Apr 12 01:43:25 user devstack@n-api.service[72504]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Apr 12 01:43:25 user devstack@n-api.service[72504]: *** Starting uWSGI 2.0.20-debian (64bit) on [Wed Apr 12 01:43:25 2023] *** Apr 12 01:43:25 user devstack@n-api.service[72504]: compiled with version: 11.2.0 on 21 March 2022 11:00:44 Apr 12 01:43:25 user devstack@n-api.service[72504]: os: Linux-5.15.0-67-generic #74-Ubuntu SMP Wed Feb 22 14:14:39 UTC 2023 Apr 12 01:43:25 user devstack@n-api.service[72504]: nodename: user Apr 12 01:43:25 user devstack@n-api.service[72504]: machine: x86_64 Apr 12 01:43:25 user devstack@n-api.service[72504]: clock source: unix Apr 12 01:43:25 user devstack@n-api.service[72504]: pcre jit disabled Apr 12 01:43:25 user devstack@n-api.service[72504]: detected number of CPU cores: 12 Apr 12 01:43:25 user devstack@n-api.service[72504]: current working directory: / Apr 12 01:43:25 user devstack@n-api.service[72504]: detected binary path: /usr/bin/uwsgi-core Apr 12 01:43:25 user devstack@n-api.service[72504]: your processes number limit is 63637 Apr 12 01:43:25 user devstack@n-api.service[72504]: your memory page size is 4096 bytes Apr 12 01:43:25 user devstack@n-api.service[72504]: detected max file descriptor number: 2048 Apr 12 01:43:25 user devstack@n-api.service[72504]: lock engine: pthread robust mutexes Apr 12 01:43:25 user devstack@n-api.service[72504]: thunder lock: enabled Apr 12 01:43:25 user devstack@n-api.service[72504]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Apr 12 01:43:25 user devstack@n-api.service[72504]: Python version: 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] Apr 12 01:43:25 user devstack@n-api.service[72504]: Python main interpreter initialized at 0x55e5ee2ad990 Apr 12 01:43:25 user devstack@n-api.service[72504]: python threads support enabled Apr 12 01:43:25 user devstack@n-api.service[72504]: your server socket listen backlog is limited to 100 connections Apr 12 01:43:25 user devstack@n-api.service[72504]: your mercy for graceful operations on workers is 90 seconds Apr 12 01:43:25 user devstack@n-api.service[72504]: mapped 537436 bytes (524 KB) for 3 cores Apr 12 01:43:25 user devstack@n-api.service[72504]: *** Operational MODE: preforking *** Apr 12 01:43:25 user devstack@n-api.service[72504]: *** uWSGI is running in multiple interpreter mode *** Apr 12 01:43:25 user devstack@n-api.service[72504]: spawned uWSGI master process (pid: 72504) Apr 12 01:43:25 user devstack@n-api.service[72504]: spawned uWSGI worker 1 (pid: 72505, cores: 1) Apr 12 01:43:25 user devstack@n-api.service[72504]: spawned uWSGI worker 2 (pid: 72506, cores: 1) Apr 12 01:43:25 user devstack@n-api.service[72504]: spawned uWSGI worker 3 (pid: 72507, cores: 1) Apr 12 01:43:25 user devstack@n-api.service[72504]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Apr 12 01:43:25 user devstack@n-api.service[72505]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Apr 12 01:43:25 user devstack@n-api.service[72507]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Apr 12 01:43:25 user devstack@n-api.service[72506]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = libvirt.LibvirtDriver {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] console_host = user {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = ext4 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] host = user {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.0.0.210 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.0.0.210 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = ps2mouse {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = libvirt.LibvirtDriver {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] console_host = user {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = ext4 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2589}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2590}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2591}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2592}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2594}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] host = user {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = libvirt.LibvirtDriver {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] console_host = user {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = ext4 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] host = user {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_wsrep_sync_wait = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.0.0.210 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.0.0.210 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = ps2mouse {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_wsrep_sync_wait = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.0.0.210 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.0.0.210 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = ps2mouse {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2602}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = custom {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: WARNING oslo_config.cfg [-] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt". Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = ['Nehalem'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_high = performance {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_low = powersave {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management_strategy = cpu_state {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] compute.vmdk_allowed_types = ['streamOptimized', 'monolithicSparse'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_wsrep_sync_wait = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: WARNING oslo_config.cfg [-] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal ( Apr 12 01:43:29 user devstack@n-api.service[72505]: live_migration_uri is deprecated for removal in favor of two other options that Apr 12 01:43:29 user devstack@n-api.service[72505]: allow to change live migration scheme and target URI: ``live_migration_scheme`` Apr 12 01:43:29 user devstack@n-api.service[72505]: and ``live_migration_inbound_addr`` respectively. Apr 12 01:43:29 user devstack@n-api.service[72505]: ). Its value may be silently ignored in the future. Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = qemu+ssh://stack@%s/system {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_wsrep_sync_wait = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_wsrep_sync_wait = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_wsrep_sync_wait = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_in_placement = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = custom {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: WARNING oslo_config.cfg [-] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt". Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = ['Nehalem'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_high = performance {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_low = powersave {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management_strategy = cpu_state {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.image_compression = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.jpeg_compression = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.playback_compression = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.streaming_mode = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] spice.zlib_compression = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: WARNING oslo_config.cfg [-] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal ( Apr 12 01:43:29 user devstack@n-api.service[72507]: live_migration_uri is deprecated for removal in favor of two other options that Apr 12 01:43:29 user devstack@n-api.service[72507]: allow to change live migration scheme and target URI: ``live_migration_scheme`` Apr 12 01:43:29 user devstack@n-api.service[72507]: and ``live_migration_inbound_addr`` respectively. Apr 12 01:43:29 user devstack@n-api.service[72507]: ). Its value may be silently ignored in the future. Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = qemu+ssh://stack@%s/system {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_at_startup = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = custom {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: WARNING oslo_config.cfg [-] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt". Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = ['Nehalem'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_high = performance {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_governor_low = powersave {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_power_management_strategy = cpu_state {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: WARNING oslo_config.cfg [-] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal ( Apr 12 01:43:29 user devstack@n-api.service[72506]: live_migration_uri is deprecated for removal in favor of two other options that Apr 12 01:43:29 user devstack@n-api.service[72506]: allow to change live migration scheme and target URI: ``live_migration_scheme`` Apr 12 01:43:29 user devstack@n-api.service[72506]: and ``live_migration_inbound_addr`` respectively. Apr 12 01:43:29 user devstack@n-api.service[72506]: ). Its value may be silently ignored in the future. Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = qemu+ssh://stack@%s/system {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_in_placement = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72505) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.image_compression = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.jpeg_compression = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.playback_compression = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.streaming_mode = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] spice.zlib_compression = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_at_startup = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_in_placement = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.image_compression = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.jpeg_compression = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.playback_compression = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.streaming_mode = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] spice.zlib_compression = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72507) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_at_startup = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: INFO dbcounter [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Registered counter for database nova_api Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_db.sqlalchemy.engines [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72505) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2609}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=72506) log_opt_values /usr/local/lib/python3.10/dist-packages/oslo_config/cfg.py:2613}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writer thread running {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72507]: INFO dbcounter [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Registered counter for database nova_api Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_db.sqlalchemy.engines [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72507) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG nova.context [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f163be6-585e-46c9-b7d4-9054f44c8b1b(cell1) {{(pid=72505) load_cells /opt/stack/nova/nova/context.py:464}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writer thread running {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72505]: INFO dbcounter [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72505]: INFO dbcounter [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Registered counter for database nova_cell1 Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_db.sqlalchemy.engines [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72505) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_db.sqlalchemy.engines [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72505) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writer thread running {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writer thread running {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: INFO dbcounter [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Registered counter for database nova_api Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG nova.context [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f163be6-585e-46c9-b7d4-9054f44c8b1b(cell1) {{(pid=72507) load_cells /opt/stack/nova/nova/context.py:464}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72507]: INFO dbcounter [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72507]: INFO dbcounter [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Registered counter for database nova_cell1 Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_db.sqlalchemy.engines [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72507) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writer thread running {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_db.sqlalchemy.engines [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72507) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_db.sqlalchemy.engines [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72506) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writer thread running {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72505]: INFO dbcounter [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG oslo_db.sqlalchemy.engines [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72505) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writer thread running {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG nova.context [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3f163be6-585e-46c9-b7d4-9054f44c8b1b(cell1) {{(pid=72506) load_cells /opt/stack/nova/nova/context.py:464}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:43:29 user devstack@n-api.service[72506]: INFO dbcounter [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72506]: INFO dbcounter [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Registered counter for database nova_cell1 Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writer thread running {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_db.sqlalchemy.engines [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72506) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72507]: INFO dbcounter [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writer thread running {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_db.sqlalchemy.engines [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72506) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG oslo_db.sqlalchemy.engines [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72507) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writer thread running {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writer thread running {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:29 user devstack@n-api.service[72506]: INFO dbcounter [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Registered counter for database nova_cell0 Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG oslo_db.sqlalchemy.engines [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=72506) _check_effective_sql_mode /usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/engines.py:335}} Apr 12 01:43:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writer thread running {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:102}} Apr 12 01:43:30 user devstack@n-api.service[72505]: WARNING keystonemiddleware.auth_token [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72505]: WARNING keystonemiddleware.auth_token [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72506]: WARNING keystonemiddleware.auth_token [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72506]: WARNING keystonemiddleware.auth_token [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72507]: WARNING keystonemiddleware.auth_token [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72507]: WARNING keystonemiddleware.auth_token [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72505]: WARNING keystonemiddleware.auth_token [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72505]: WARNING keystonemiddleware.auth_token [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72505]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e5ee2ad990 pid: 72505 (default app) Apr 12 01:43:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:43:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] 10.0.0.210 "GET /compute/v2.1/" status: 200 len: 387 microversion: 2.1 time: 0.001462 Apr 12 01:43:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 1/1] 10.0.0.210 () {50 vars in 783 bytes} [Wed Apr 12 01:43:30 2023] GET /compute/v2.1/ => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:43:30 user devstack@n-api.service[72507]: WARNING keystonemiddleware.auth_token [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72507]: WARNING keystonemiddleware.auth_token [None req-3d1c7f76-9b3f-4c9e-b531-197869cbcddf None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72507]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e5ee2ad990 pid: 72507 (default app) Apr 12 01:43:30 user devstack@n-api.service[72506]: WARNING keystonemiddleware.auth_token [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Apr 12 01:43:30 user devstack@n-api.service[72506]: WARNING keystonemiddleware.auth_token [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Apr 12 01:43:30 user devstack@n-api.service[72506]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e5ee2ad990 pid: 72506 (default app) Apr 12 01:45:39 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3fa4896b-e64a-4782-945a-0c83450b7f7c None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001610 Apr 12 01:45:39 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 2/2] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:38 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:39 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:39 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:39 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:39 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3fab3078-a266-462c-8ac6-7e033726dc03 admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3fab3078-a266-462c-8ac6-7e033726dc03 admin admin] 10.0.0.210 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.375721 Apr 12 01:45:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 1/3] 10.0.0.210 () {60 vars in 1285 bytes} [Wed Apr 12 01:45:39 2023] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.003276 Apr 12 01:45:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 1/4] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:41 2023] GET /compute/v2.1 => generated 387 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f8948de5-b615-48fd-ad6a-3c88dd838aee admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "c1", "disk": 1, "swap": 0, "rxtx_factor": 1.0, "ram": 256, "vcpus": 1, "name": "cirros256", "os-flavor-access:is_public": true}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f8948de5-b615-48fd-ad6a-3c88dd838aee admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.384744 Apr 12 01:45:41 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 3/5] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:41 2023] POST /compute/v2.1/flavors => generated 405 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b1e5bc8a-8dbd-4c39-a528-b016fea3a509 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b1e5bc8a-8dbd-4c39-a528-b016fea3a509 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.051595 Apr 12 01:45:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 2/6] 10.0.0.210 () {64 vars in 1329 bytes} [Wed Apr 12 01:45:41 2023] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001431 Apr 12 01:45:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 2/7] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:43 2023] GET /compute/v2.1 => generated 387 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:43 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ab55e1de-b2a1-442a-8cba-3d0812d84f1f admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "id": "d1", "name": "ds512M", "rxtx_factor": 1.0, "vcpus": 1, "disk": 5, "ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ab55e1de-b2a1-442a-8cba-3d0812d84f1f admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.067085 Apr 12 01:45:43 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 4/8] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:43 2023] POST /compute/v2.1/flavors => generated 402 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-31c21a22-9609-4d93-a22f-de8804caede3 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-31c21a22-9609-4d93-a22f-de8804caede3 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026574 Apr 12 01:45:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 3/9] 10.0.0.210 () {64 vars in 1329 bytes} [Wed Apr 12 01:45:43 2023] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001804 Apr 12 01:45:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 3/10] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:45 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-bc19809d-b149-4276-93e1-f6e440104b2e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "id": "d2", "vcpus": 1, "rxtx_factor": 1.0, "swap": 0, "disk": 10, "ram": 1024}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:45 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-bc19809d-b149-4276-93e1-f6e440104b2e admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.062582 Apr 12 01:45:45 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 5/11] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:45 2023] POST /compute/v2.1/flavors => generated 402 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-c9f7f181-b60a-4390-966e-33878c820a49 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-c9f7f181-b60a-4390-966e-33878c820a49 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036767 Apr 12 01:45:45 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 4/12] 10.0.0.210 () {64 vars in 1329 bytes} [Wed Apr 12 01:45:45 2023] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001472 Apr 12 01:45:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 4/13] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:47 2023] GET /compute/v2.1 => generated 387 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ad3ccb68-8ac7-4969-ae20-7f7f3cb51d96 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "name": "ds2G", "rxtx_factor": 1.0, "id": "d3", "ram": 2048, "os-flavor-access:is_public": true, "disk": 10, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ad3ccb68-8ac7-4969-ae20-7f7f3cb51d96 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.056751 Apr 12 01:45:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 6/14] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:47 2023] POST /compute/v2.1/flavors => generated 402 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1ea9d75f-6afb-463d-b10a-fd7d04cea24d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:47 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1ea9d75f-6afb-463d-b10a-fd7d04cea24d admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025180 Apr 12 01:45:47 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 5/15] 10.0.0.210 () {64 vars in 1329 bytes} [Wed Apr 12 01:45:47 2023] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.002351 Apr 12 01:45:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 5/16] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:49 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:49 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-703e1751-65d2-4fb4-8de9-4ad229d842a0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "rxtx_factor": 1.0, "name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4", "ram": 4096, "swap": 0, "disk": 20, "vcpus": 4}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:49 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-703e1751-65d2-4fb4-8de9-4ad229d842a0 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.062253 Apr 12 01:45:49 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 7/17] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:49 2023] POST /compute/v2.1/flavors => generated 402 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025008 Apr 12 01:45:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 6/18] 10.0.0.210 () {64 vars in 1329 bytes} [Wed Apr 12 01:45:49 2023] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:51 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:51 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4f9ccde-2937-4b9d-a7e6-03bc294600f2 None None] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001688 Apr 12 01:45:51 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 6/19] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:51 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1112ea4a-2f4f-4afc-bdfd-6fb4cd67e4a5 admin admin] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1112ea4a-2f4f-4afc-bdfd-6fb4cd67e4a5 admin admin] 10.0.0.210 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2096 microversion: 2.61 time: 0.093238 Apr 12 01:45:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 8/20] 10.0.0.210 () {60 vars in 1285 bytes} [Wed Apr 12 01:45:51 2023] GET /compute/v2.1/flavors/detail?is_public=True => generated 2096 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:45:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.002437 Apr 12 01:45:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 7/21] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:53 2023] GET /compute/v2.1 => generated 387 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:45:54 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8c3d7ab3-57c6-4347-a919-fc1f9b5eb0ff admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "ram": 512, "disk": 1, "vcpus": 1, "swap": 0, "name": "m1.tiny", "id": "1", "os-flavor-access:is_public": true}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:54 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8c3d7ab3-57c6-4347-a919-fc1f9b5eb0ff admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 400 microversion: 2.61 time: 0.371013 Apr 12 01:45:54 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 7/22] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:53 2023] POST /compute/v2.1/flavors => generated 400 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c2031096-3bbe-4b26-8d7b-92fff9d1e888 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c2031096-3bbe-4b26-8d7b-92fff9d1e888 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.048937 Apr 12 01:45:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 9/23] 10.0.0.210 () {64 vars in 1326 bytes} [Wed Apr 12 01:45:54 2023] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:56 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:56 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.002253 Apr 12 01:45:56 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 8/24] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:56 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6626ffc9-9346-43ae-a771-5159477e9803 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "name": "m1.small", "vcpus": 1, "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 2048, "rxtx_factor": 1.0, "disk": 20}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6626ffc9-9346-43ae-a771-5159477e9803 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 403 microversion: 2.61 time: 0.090569 Apr 12 01:45:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 8/25] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:56 2023] POST /compute/v2.1/flavors => generated 403 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Apr 12 01:45:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-77453923-1a56-4764-a341-a91d7d230380 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-77453923-1a56-4764-a341-a91d7d230380 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029201 Apr 12 01:45:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 10/26] 10.0.0.210 () {64 vars in 1326 bytes} [Wed Apr 12 01:45:56 2023] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:45:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:45:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.002078 Apr 12 01:45:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 9/27] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:45:58 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:45:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d870730a-879b-479a-8f9b-546bb086c5bf admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.medium", "vcpus": 2, "id": "3", "disk": 40, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d870730a-879b-479a-8f9b-546bb086c5bf admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.064552 Apr 12 01:45:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 9/28] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:45:58 2023] POST /compute/v2.1/flavors => generated 404 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:45:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d0055635-1c4c-442e-8a12-e763c829ce6d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:45:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d0055635-1c4c-442e-8a12-e763c829ce6d admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.026515 Apr 12 01:45:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 11/29] 10.0.0.210 () {64 vars in 1326 bytes} [Wed Apr 12 01:45:58 2023] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:46:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:46:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001378 Apr 12 01:46:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 10/30] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:46:00 2023] GET /compute/v2.1 => generated 387 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:46:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b16ba8b5-a910-45df-b594-7320dcdaa163 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 80, "rxtx_factor": 1.0, "ram": 8192, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "vcpus": 4, "name": "m1.large", "id": "4"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:46:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b16ba8b5-a910-45df-b594-7320dcdaa163 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 403 microversion: 2.61 time: 0.089245 Apr 12 01:46:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 10/31] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:46:00 2023] POST /compute/v2.1/flavors => generated 403 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:46:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f7d0e832-389d-4744-b70b-527ec3463e3c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:46:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f7d0e832-389d-4744-b70b-527ec3463e3c admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.024692 Apr 12 01:46:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 12/32] 10.0.0.210 () {64 vars in 1326 bytes} [Wed Apr 12 01:46:00 2023] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:46:02 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:46:02 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001364 Apr 12 01:46:02 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 11/33] 10.0.0.210 () {54 vars in 924 bytes} [Wed Apr 12 01:46:02 2023] GET /compute/v2.1 => generated 387 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:46:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-11997baa-8cda-434d-a169-bd5189423b32 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "5", "name": "m1.xlarge", "disk": 160, "swap": 0, "vcpus": 8, "ram": 16384}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:46:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-11997baa-8cda-434d-a169-bd5189423b32 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.073948 Apr 12 01:46:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 11/34] 10.0.0.210 () {64 vars in 1276 bytes} [Wed Apr 12 01:46:02 2023] POST /compute/v2.1/flavors => generated 406 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:46:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-34ef9112-fd22-4cc8-969f-9061e16fd6dc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:46:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-34ef9112-fd22-4cc8-969f-9061e16fd6dc admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.024159 Apr 12 01:46:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 13/35] 10.0.0.210 () {64 vars in 1326 bytes} [Wed Apr 12 01:46:02 2023] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:46:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:46:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bd4f463a-e46c-4028-bb4c-5130b7aac06c admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.001387 Apr 12 01:46:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 12/36] 10.0.0.210 () {58 vars in 1006 bytes} [Wed Apr 12 01:46:33 2023] GET /compute/v2.1 => generated 387 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=15,nova_api:INSERT=5 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:46:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:46:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] 10.0.0.210 "GET /compute/v2.1/os-services?binary=nova-compute&host=user" status: 200 len: 247 microversion: 2.69 time: 0.094843 Apr 12 01:46:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 12/37] 10.0.0.210 () {64 vars in 1388 bytes} [Wed Apr 12 01:46:33 2023] GET /compute/v2.1/os-services?binary=nova-compute&host=user => generated 247 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:47:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-34ef9112-fd22-4cc8-969f-9061e16fd6dc admin admin] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:47:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-34ef9112-fd22-4cc8-969f-9061e16fd6dc admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.004228 Apr 12 01:47:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 14/38] 10.0.0.210 () {58 vars in 1006 bytes} [Wed Apr 12 01:47:05 2023] GET /compute/v2.1 => generated 387 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:47:05 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=32,nova_api:SAVEPOINT=5,nova_api:INSERT=5,nova_api:RELEASE=5 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:47:05 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4f591d62-a617-495b-beef-c6d31f2a977a admin admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:47:05 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4f591d62-a617-495b-beef-c6d31f2a977a admin admin] 10.0.0.210 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4182 microversion: 2.61 time: 0.068353 Apr 12 01:47:05 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 13/39] 10.0.0.210 () {64 vars in 1367 bytes} [Wed Apr 12 01:47:05 2023] GET /compute/v2.1/flavors/detail?is_public=True => generated 4182 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:47:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:47:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.002074 Apr 12 01:47:09 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 13/40] 10.0.0.210 () {58 vars in 1006 bytes} [Wed Apr 12 01:47:09 2023] GET /compute/v2.1 => generated 387 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:47:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=10,nova_api:SAVEPOINT=5,nova_api:RELEASE=5 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:47:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-08f17c0b-99f6-4b56-8a87-a503f11d5fd1 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 128, "rxtx_factor": 1.0, "id": "42", "vcpus": 1, "disk": 1, "os-flavor-access:is_public": true, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.nano"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:47:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-08f17c0b-99f6-4b56-8a87-a503f11d5fd1 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 403 microversion: 2.61 time: 0.065714 Apr 12 01:47:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 15/41] 10.0.0.210 () {68 vars in 1358 bytes} [Wed Apr 12 01:47:09 2023] POST /compute/v2.1/flavors => generated 403 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:47:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-60e97328-da52-4c05-8955-11800ff93faf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:47:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-60e97328-da52-4c05-8955-11800ff93faf admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025802 Apr 12 01:47:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 14/42] 10.0.0.210 () {68 vars in 1411 bytes} [Wed Apr 12 01:47:09 2023] POST /compute/v2.1/flavors/42/os-extra_specs => generated 43 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:47:13 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:47:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4eb4661-6a00-43a5-a104-2dd5345cedfc admin admin] 10.0.0.210 "GET /compute/v2.1" status: 200 len: 387 microversion: 2.1 time: 0.004167 Apr 12 01:47:13 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 14/43] 10.0.0.210 () {58 vars in 1006 bytes} [Wed Apr 12 01:47:13 2023] GET /compute/v2.1 => generated 387 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:47:13 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-16f348f5-8cc1-4a2b-9f70-ef0ab6bd3745 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 192, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.micro", "vcpus": 1, "rxtx_factor": 1.0, "swap": 0, "disk": 1, "id": "84", "os-flavor-access:is_public": true}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:47:13 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-16f348f5-8cc1-4a2b-9f70-ef0ab6bd3745 admin admin] 10.0.0.210 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.066592 Apr 12 01:47:13 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 16/44] 10.0.0.210 () {68 vars in 1358 bytes} [Wed Apr 12 01:47:13 2023] POST /compute/v2.1/flavors => generated 404 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:47:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3bec4509-0771-46ba-bc0a-76e1855834fd admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:47:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3bec4509-0771-46ba-bc0a-76e1855834fd admin admin] 10.0.0.210 "POST /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.025136 Apr 12 01:47:13 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 15/45] 10.0.0.210 () {68 vars in 1411 bytes} [Wed Apr 12 01:47:13 2023] POST /compute/v2.1/flavors/84/os-extra_specs => generated 43 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:49:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:49:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:49:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-079c1413-b102-4ddc-87cf-1803881e77d8 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Action: 'create', calling method: .version_select at 0x7fb2040d5ab0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-355677210", "metadata": {"Type": "work"}}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-079c1413-b102-4ddc-87cf-1803881e77d8 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 1.878353 Apr 12 01:49:27 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 15/46] 10.0.0.210 () {64 vars in 1233 bytes} [Wed Apr 12 01:49:25 2023] POST /compute/v2.1/os-volumes => generated 352 bytes in 1880 msecs (HTTP/1.1 200) 10 headers in 447 bytes (1 switches on core 0) Apr 12 01:49:27 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6,nova_api:INSERT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:49:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-17a0b40a-d4d2-47eb-aed6-b3205943809a tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb204075240>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-17a0b40a-d4d2-47eb-aed6-b3205943809a tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 200 len: 352 microversion: 2.1 time: 0.366543 Apr 12 01:49:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 17/47] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:27 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 352 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=2,nova_api:RELEASE=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:49:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f02c356b-4f90-4194-9405-1f022ff30975 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb20415fac0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f02c356b-4f90-4194-9405-1f022ff30975 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 200 len: 353 microversion: 2.1 time: 0.189269 Apr 12 01:49:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 16/48] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:29 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 353 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8b02dcf5-5223-459f-81ab-06681ff7e947 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb203f5ef80>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8b02dcf5-5223-459f-81ab-06681ff7e947 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 200 len: 353 microversion: 2.1 time: 0.051907 Apr 12 01:49:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 16/49] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:29 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 353 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9ee02dc3-efc0-40b8-ab30-047f54721c51 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb203f03880>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9ee02dc3-efc0-40b8-ab30-047f54721c51 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 202 len: 0 microversion: 2.1 time: 0.114025 Apr 12 01:49:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 18/50] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 01:49:29 2023] DELETE /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:49:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-de0ccc59-6129-49ed-bef9-1e6529c7c4a5 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb2040ef490>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-de0ccc59-6129-49ed-bef9-1e6529c7c4a5 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 200 len: 352 microversion: 2.1 time: 0.067585 Apr 12 01:49:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 17/51] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:29 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 352 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:30 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4578a4bd-3fb9-4323-b373-25d1c39613f3 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb203f5f1c0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4578a4bd-3fb9-4323-b373-25d1c39613f3 tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 200 len: 352 microversion: 2.1 time: 0.070426 Apr 12 01:49:30 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 17/52] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:30 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 352 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0291a2af-7259-4718-a134-67adde8ddd6d tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Calling method '.version_select at 0x7fb203f03f40>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-0291a2af-7259-4718-a134-67adde8ddd6d tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] HTTP exception thrown: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. Apr 12 01:49:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0291a2af-7259-4718-a134-67adde8ddd6d tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Request method failure captured: Apr 12 01:49:31 user devstack@n-api.service[72505]: request: GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 HTTP/1.1 Apr 12 01:49:31 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:49:31 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:49:31 user devstack@n-api.service[72505]: Connection: close Apr 12 01:49:31 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:49:31 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:49:31 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:49:31 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:49:31 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg4lcmtmKUPCpXkPnjdUdADXmwHpNYCpJTrkkpQIDZVcvBM2Wuij7-R8AJWQLm1S-eRWrpEENHg9kBy_iesrDFoOyj3Ozw5C6S89U76YxQr51pZPBU6JXjMFEp_zSM5PF9GrfYg7y7d4uKCSDj4aQ2f8Kk0a2o5NRlE6PegmcP-GVaY Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Project-Id: 4ab1ed07b27f438aa1fdc1c4c2d36fb4 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesGetTestJSON-231808491 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/4ab1ed07b27f438aa1fdc1c4c2d36fb4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/4ab1ed07b27f438aa1fdc1c4c2d36fb4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/4ab1ed07b27f438aa1fdc1c4c2d36fb4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesGetTestJSON-231808491 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Tenant-Id: 4ab1ed07b27f438aa1fdc1c4c2d36fb4 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesGetTestJSON-231808491 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-User: tempest-VolumesGetTestJSON-231808491-project-member Apr 12 01:49:31 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:49:31 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:49:31 user devstack@n-api.service[72505]: X-User-Id: dd1872a9a3d5429ba7abd53f612e3467 Apr 12 01:49:31 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesGetTestJSON-231808491-project-member Apr 12 01:49:31 user devstack@n-api.service[72505]: method: Apr 12 01:49:31 user devstack@n-api.service[72505]: exception: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. Apr 12 01:49:31 user devstack@n-api.service[72505]: action_args: {'id': '85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3'} Apr 12 01:49:31 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. (HTTP 404) (Request-ID: req-6cb26dda-0f14-45fd-9238-1fb85358cd56) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. Apr 12 01:49:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:49:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0291a2af-7259-4718-a134-67adde8ddd6d tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] Returning 404 to user: Volume 85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:49:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0291a2af-7259-4718-a134-67adde8ddd6d tempest-VolumesGetTestJSON-231808491 tempest-VolumesGetTestJSON-231808491-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3" status: 404 len: 109 microversion: 2.1 time: 0.040885 Apr 12 01:49:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 19/53] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 01:49:31 2023] GET /compute/v2.1/os-volumes/85e27498-fdd3-4fc5-bdd9-7d1e46fb8bd3 => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:49:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8aac8e69-631c-44ad-af60-51c9e5e4223f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: .version_select at 0x7fb2040efac0>, body: {"keypair": {"name": "tempest-keypair-1931079911", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE1sUn4ZyIzVWiCtOrmiSV/0bbq9mYMTAsy+xwTDk6JQuJ5RMkrEvjPYuyz4Zno9RvobzKj4DM8GWPUelp3zq8n5Xrf63oYfGsblJs4n7uxBlQIBcuWW0l1ExcediwSzPw=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:39 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-8aac8e69-631c-44ad-af60-51c9e5e4223f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:39 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-8aac8e69-631c-44ad-af60-51c9e5e4223f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-8aac8e69-631c-44ad-af60-51c9e5e4223f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.247495 Apr 12 01:49:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 18/54] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:49:39 2023] POST /compute/v2.1/os-keypairs => generated 386 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-722086832", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:45 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] validate_networks() for [('babcad8e-dca2-47b4-80af-a9e0cede0853', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f7fbfd4d-b6d4-4682-b6a9-b4bf77987c47 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: .version_select at 0x7fb203f58f70>, body: {"keypair": {"name": "tempest-keypair-663896141", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLr2NTbVfrYVAmPq/WSiXNxhw73D4U473knUSj3S2SzGG67zJUMcYD7NdTMJnRymKmrJWRxrjStG5itGjTKaX5Cd+wZw4hLfWVikFnVEIrbrRCuVHjjCnBxwGGHScYXLHw=="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-f7fbfd4d-b6d4-4682-b6a9-b4bf77987c47 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-f7fbfd4d-b6d4-4682-b6a9-b4bf77987c47 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for user d2ca4855c3e9461cb27c88469086d7d8 and project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-213236224", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "16f5655f-6169-43e9-9110-9fcbab7a3d5d"}], "security_groups": [{"name": "tempest-securitygroup--1634025199"}], "key_name": "tempest-keypair-1931079911", "user_data": "}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f7fbfd4d-b6d4-4682-b6a9-b4bf77987c47 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.286985 Apr 12 01:49:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 20/55] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:49:46 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:47 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] validate_networks() for [('16f5655f-6169-43e9-9110-9fcbab7a3d5d', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for user 889189da060e4b54a48ce9a013e04a3d and project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-55be52d8-8df5-4d9d-b6d2-3ee663837b40 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 381 microversion: 2.1 time: 2.673786 Apr 12 01:49:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 18/56] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:45 2023] POST /compute/v2.1/servers => generated 381 bytes in 2677 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-918eedf8-8594-4740-885e-dc308eba8cc3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:47 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-918eedf8-8594-4740-885e-dc308eba8cc3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:47 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-918eedf8-8594-4740-885e-dc308eba8cc3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-918eedf8-8594-4740-885e-dc308eba8cc3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1215 microversion: 2.1 time: 0.171140 Apr 12 01:49:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 21/57] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:47 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1215 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-808158946", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:48 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] validate_networks() for [('babcad8e-dca2-47b4-80af-a9e0cede0853', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-49623664", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:48 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] validate_networks() for [('1d227a8b-1ad3-4bc6-a32e-4322c8f75eba', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5108f3d1-83f8-4e4f-b985-01ab8e8070b5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 2.493143 Apr 12 01:49:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 19/58] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:46 2023] POST /compute/v2.1/servers => generated 407 bytes in 2494 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d385b192-5885-4cd3-b0ec-4abc5e08f85e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d385b192-5885-4cd3-b0ec-4abc5e08f85e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-d385b192-5885-4cd3-b0ec-4abc5e08f85e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d385b192-5885-4cd3-b0ec-4abc5e08f85e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1232 microversion: 2.1 time: 0.235038 Apr 12 01:49:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 20/59] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:49 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1232 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for user 889189da060e4b54a48ce9a013e04a3d and project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:49 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-480eb4dd-c9a6-426b-a916-341472b0def0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 381 microversion: 2.1 time: 1.540534 Apr 12 01:49:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 19/60] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:48 2023] POST /compute/v2.1/servers => generated 381 bytes in 1543 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f8a79a96-d83c-4eb6-abe3-04d6660009cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f8a79a96-d83c-4eb6-abe3-04d6660009cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:50 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f8a79a96-d83c-4eb6-abe3-04d6660009cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f8a79a96-d83c-4eb6-abe3-04d6660009cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 200 len: 1215 microversion: 2.1 time: 0.397550 Apr 12 01:49:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 21/61] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:49 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 1215 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Getting quotas for project 021f9aee2b7849ef8fcd67d6d6522d27. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Getting quotas for user 728c534ef1f54c50ad2b619b44c3b9c7 and project 021f9aee2b7849ef8fcd67d6d6522d27. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:50 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-81468de8-3320-46a8-bcdd-4f7f20fbd95b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 1.896931 Apr 12 01:49:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 22/62] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:48 2023] POST /compute/v2.1/servers => generated 378 bytes in 1899 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-044774b9-4c01-4905-ae8d-8d72b0234cc9 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-044774b9-4c01-4905-ae8d-8d72b0234cc9 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:50 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-044774b9-4c01-4905-ae8d-8d72b0234cc9 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-044774b9-4c01-4905-ae8d-8d72b0234cc9 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1209 microversion: 2.1 time: 0.168066 Apr 12 01:49:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 20/63] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:50 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1209 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:51 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-494306683", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "0ef60c4f-0414-4002-adfd-b2fd10b1cd53"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:51 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] validate_networks() for [('0ef60c4f-0414-4002-adfd-b2fd10b1cd53', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Getting quotas for project 2a7713a76bb546d2a348927f18b12eb7. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Getting quotas for user 6c503a0141cf4794bf4e6faba59bdba3 and project 2a7713a76bb546d2a348927f18b12eb7. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:52 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7527f881-3466-43ff-a80d-12488526a26d tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 2.029053 Apr 12 01:49:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 22/64] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:50 2023] POST /compute/v2.1/servers => generated 378 bytes in 2032 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8b63a6eb-f96d-49c7-a7c4-96b0fe3a1bf2 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:52 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8b63a6eb-f96d-49c7-a7c4-96b0fe3a1bf2 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:53 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-8b63a6eb-f96d-49c7-a7c4-96b0fe3a1bf2 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c503a0141cf4794bf4e6faba59bdba3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a7713a76bb546d2a348927f18b12eb7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:53 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8b63a6eb-f96d-49c7-a7c4-96b0fe3a1bf2 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 200 len: 1208 microversion: 2.1 time: 0.233873 Apr 12 01:49:53 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 23/65] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:52 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 1208 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-682748151", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "10203cb1-b210-4779-b45b-c0822f9562cb"}], "security_groups": [{"name": "tempest-securitygroup--1979274568"}], "key_name": "tempest-keypair-663896141", "user_data": "}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:53 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] validate_networks() for [('10203cb1-b210-4779-b45b-c0822f9562cb', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for user d2ca4855c3e9461cb27c88469086d7d8 and project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:49:55 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:49:55 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-faba2212-4861-40c4-8865-356c433f227e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 1.779743 Apr 12 01:49:55 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 21/66] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:49:53 2023] POST /compute/v2.1/servers => generated 400 bytes in 1781 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:49:55 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1c91e62a-e626-4a98-8521-b955373d0754 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:49:55 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1c91e62a-e626-4a98-8521-b955373d0754 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:49:55 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-1c91e62a-e626-4a98-8521-b955373d0754 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:49:55 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1c91e62a-e626-4a98-8521-b955373d0754 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1230 microversion: 2.1 time: 0.134242 Apr 12 01:49:55 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 23/67] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:49:55 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1230 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Creating event network-changed:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Acquiring lock "compute-rpcapi-router" {{(pid=72505) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c2a93990-10c5-44fa-9a29-7c141bb765db req-b0cab9a8-be94-4826-a481-6311394f755d service nova] Acquired lock "compute-rpcapi-router" {{(pid=72505) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72505]: INFO nova.compute.rpcapi [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 Apr 12 01:49:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] Releasing lock "compute-rpcapi-router" {{(pid=72505) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} Apr 12 01:49:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-785f4c72-b075-4774-925c-6bf0dc67bb74 None None] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.162641 Apr 12 01:49:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 24/68] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:49:56 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Creating event network-changed:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Acquiring lock "compute-rpcapi-router" {{(pid=72506) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cea7c8d1-4c61-4bcd-aa4d-ef423e185f00 req-0faed6c6-482f-4df6-8674-c0535a9f0bdc service nova] Acquired lock "compute-rpcapi-router" {{(pid=72506) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:56 user devstack@n-api.service[72506]: INFO nova.compute.rpcapi [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Automatically selected compute RPC version 6.2 from minimum service version 66 Apr 12 01:49:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] Releasing lock "compute-rpcapi-router" {{(pid=72506) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} Apr 12 01:49:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-add472a9-db65-4549-89a9-d43610d22a3d None None] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.186127 Apr 12 01:49:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 22/69] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:49:56 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Creating event network-changed:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=72507) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:312}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d5669752-d3a4-4357-9d04-519b9f6ce224 req-34edeaed-b025-4b7e-83b9-2d4bd9c4a4a8 service nova] Acquired lock "compute-rpcapi-router" {{(pid=72507) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:315}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:57 user devstack@n-api.service[72507]: INFO nova.compute.rpcapi [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Automatically selected compute RPC version 6.2 from minimum service version 66 Apr 12 01:49:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] Releasing lock "compute-rpcapi-router" {{(pid=72507) lock /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:333}} Apr 12 01:49:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-9f1d6255-eb60-4c3b-9cd2-13e24ad2d945 None None] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.157816 Apr 12 01:49:57 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 24/70] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:49:56 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:59 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:49:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:49:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:49:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:49:59 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] Creating event network-changed:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:49:59 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-9e87c216-6a74-4cd9-affe-0c7db18c7950 req-fb71d2e6-3338-49b6-b9c3-da3454ce7042 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038609 Apr 12 01:49:59 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 23/71] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:49:59 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:49:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] Creating event network-changed:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:49:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8c809fa5-dca2-4b7a-8ba5-d52d15e40209 req-47ec39f9-8210-4646-9083-a2dcd02ec94d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066033 Apr 12 01:49:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 25/72] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:49:59 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] Creating event network-changed:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:50:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7ee42332-2c70-4798-8827-cb08e86ca8c6 req-a6d3545d-7492-4621-9e1d-b83b2e887b3a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036607 Apr 12 01:50:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 25/73] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:01 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "name": "network-vif-plugged", "status": "completed", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:05 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=20 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] Creating event network-vif-plugged:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:50:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8c62d455-bf86-41b8-ba6c-9e0654fcbab9 req-3cba99cc-65da-4c65-82a6-2c3c3092f816 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037536 Apr 12 01:50:05 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 24/74] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:06 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:06 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:50:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-1b2e7b1f-0dc0-46a1-b92a-df4d17638c7e req-82eb23be-1b2f-4eca-bd31-dfa18ca0b0ff service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036367 Apr 12 01:50:06 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 26/75] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:06 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:06 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=22 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:06 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:50:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-96d0650b-695a-4a33-a4c0-843093430217 req-afa969a6-d85d-41c2-828c-604f858a47aa service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035655 Apr 12 01:50:06 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 26/76] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:06 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "name": "network-vif-plugged", "status": "completed", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}, {"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}, {"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Creating event network-vif-plugged:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:50:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:50:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:50:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-610adcc9-eedc-467e-ba3e-21a6984c88ff req-78930aa9-9a5a-442f-bbf1-c5eff213708a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.062279 Apr 12 01:50:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 25/77] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:07 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:50:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-6babd888-e4e9-43c2-ad11-7bf4f7ff2abe req-49a7ce95-ca9a-4587-aa7a-3c92b9d8ca60 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043753 Apr 12 01:50:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 27/78] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:08 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e32262f9-d05c-4f67-9e75-3fcfe5b4ea74 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Action: 'create', calling method: .version_select at 0x7fb203e88700>, body: {"keypair": {"name": "tempest-keypair-38328491", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIoBx7IwA26nr+H9WbInCKQvzt885MvXO4T7OFAxKtTB1Y+/K8rC2POLNWs9WxsvQqtRVrXppKseGE6lFkO0s+3jTpEjBu+bRewGqCxddqqCMhq2Zh6JTma7wnriFenAJw=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e32262f9-d05c-4f67-9e75-3fcfe5b4ea74 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Getting quotas for project 742d0477bb3b464e8d98d42a85c330e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e32262f9-d05c-4f67-9e75-3fcfe5b4ea74 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Getting quotas for user e400a3359c844c449445cfba08ef7a7b and project 742d0477bb3b464e8d98d42a85c330e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e32262f9-d05c-4f67-9e75-3fcfe5b4ea74 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.303974 Apr 12 01:50:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 27/79] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:50:08 2023] POST /compute/v2.1/os-keypairs => generated 384 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:50:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-5906689e-5dd6-47a0-ad53-84860eb26232 req-ab7cba29-1d7b-4b52-96f1-fb4cfa849717 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044987 Apr 12 01:50:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 26/80] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:08 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "name": "network-vif-plugged", "status": "completed", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}, {"server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "name": "network-vif-plugged", "status": "completed", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Creating event network-vif-plugged:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:50:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] Creating event network-vif-plugged:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:50:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-15f15156-4680-494f-9bef-4578021d5a98 req-c571be4b-81d0-410c-8ed5-a41fb3fb0a5a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.066341 Apr 12 01:50:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 28/81] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:09 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:12 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachSCSIVolumeTestJSON-server-1735119552", "imageRef": "aaf0837b-b109-4309-87a6-5a6729f0eccc", "flavorRef": "42", "config_drive": true, "networks": [{"uuid": "e5e461dd-9a8b-4f2a-90ec-34fb659152b4"}], "security_groups": [{"name": "tempest-securitygroup--222607931"}], "key_name": "tempest-keypair-38328491", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3e872237-d65b-42cb-a3b0-9bc791fc7d75 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Action: 'create', calling method: .version_select at 0x7fb203e8a0e0>, body: {"keypair": {"name": "tempest-keypair-599771826", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDDhtyZheVbYlLeUhN71vEcEpIuCfTBPYbfrrTYkKIj9EYz55uOf6GTaC6L9Qt5AUB8U6/UJe/T4rJ4Zr50Pm9jh0coWiuaA7ovp73w+1OnKciWJB+jFAJEIEOPz+MR+bw=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:12 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-3e872237-d65b-42cb-a3b0-9bc791fc7d75 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-3e872237-d65b-42cb-a3b0-9bc791fc7d75 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for user aeb7fd40946e43dc860d63bb8958d633 and project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3e872237-d65b-42cb-a3b0-9bc791fc7d75 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.300799 Apr 12 01:50:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 28/82] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:50:12 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:12 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] validate_networks() for [('e5e461dd-9a8b-4f2a-90ec-34fb659152b4', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:50:13 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:13 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:50:13 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-2bace5fd-db3c-46a5-afaa-a62790e7541a req-d6a9ce45-2853-4bde-8101-b984dd86ce7c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116340 Apr 12 01:50:13 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 29/83] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:13 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:14 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Getting quotas for project 742d0477bb3b464e8d98d42a85c330e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:14 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Getting quotas for user e400a3359c844c449445cfba08ef7a7b and project 742d0477bb3b464e8d98d42a85c330e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:50:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='aaf0837b-b109-4309-87a6-5a6729f0eccc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:50:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-795cfe4d-3e89-4b6e-9f54-68b653d3daa0 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 1.629433 Apr 12 01:50:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 27/84] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:50:12 2023] POST /compute/v2.1/servers => generated 403 bytes in 1634 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:50:14 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5fe303af-dca9-45ce-8bc5-28b5a27c1946 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:14 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5fe303af-dca9-45ce-8bc5-28b5a27c1946 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:14 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-5fe303af-dca9-45ce-8bc5-28b5a27c1946 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e400a3359c844c449445cfba08ef7a7b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '742d0477bb3b464e8d98d42a85c330e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-622b1810-bd7b-4969-8419-9ac075b89a97 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Action: 'create', calling method: .version_select at 0x7fb203edec20>, body: {"keypair": {"name": "tempest-keypair-2091713473", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKW4t6M942/LHh6daB7qePEWtiWAJ7b1N9u1/DPWtDalmVzTtTt8SSDAJEDRAeJhrXKOzi8MtjgYWJTbhnxcGIllhlDsuyE7LR5HijyXxbU86EQHerBjvbSSrdtGqPatAQ=="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:14 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5fe303af-dca9-45ce-8bc5-28b5a27c1946 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 200 len: 1238 microversion: 2.1 time: 0.357662 Apr 12 01:50:14 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 29/85] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:14 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 1238 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:14 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-622b1810-bd7b-4969-8419-9ac075b89a97 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Getting quotas for project 3a1f516270354a82995f75c4c149dad2. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:14 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-622b1810-bd7b-4969-8419-9ac075b89a97 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Getting quotas for user 592f78cbcf7046e68bcf6c88e039a9be and project 3a1f516270354a82995f75c4c149dad2. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-622b1810-bd7b-4969-8419-9ac075b89a97 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.275716 Apr 12 01:50:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 30/86] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:50:14 2023] POST /compute/v2.1/os-keypairs => generated 386 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:15 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=21 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:50:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-9f862628-6305-488c-a805-a016c5f1fea7 req-5dc4562f-873f-4edb-9d3e-e9b82b3d2672 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.117051 Apr 12 01:50:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 28/87] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:15 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 01:50:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-1833946083", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "387dd6cf-c04c-4f01-b526-b70c95a66b22"}], "security_groups": [{"name": "tempest-securitygroup--338279859"}], "key_name": "tempest-keypair-599771826", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:17 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] validate_networks() for [('387dd6cf-c04c-4f01-b526-b70c95a66b22', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for user aeb7fd40946e43dc860d63bb8958d633 and project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:50:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:50:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeSwap-server-74378898", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "409e7f9e-7aa8-44e0-8d4f-6582d339e7c6"}], "security_groups": [{"name": "tempest-securitygroup--1711002915"}], "key_name": "tempest-keypair-2091713473", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=5 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-81b295fb-5683-4af7-8628-be50112f806f tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 1.817080 Apr 12 01:50:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 30/88] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:50:17 2023] POST /compute/v2.1/servers => generated 403 bytes in 1823 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:50:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cb97f5f4-5f52-429b-bc9c-e18a2f29b1ef tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cb97f5f4-5f52-429b-bc9c-e18a2f29b1ef tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:19 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cb97f5f4-5f52-429b-bc9c-e18a2f29b1ef tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cb97f5f4-5f52-429b-bc9c-e18a2f29b1ef tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335" status: 200 len: 1235 microversion: 2.1 time: 0.209967 Apr 12 01:50:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 29/89] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:19 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 => generated 1235 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:19 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] validate_networks() for [('409e7f9e-7aa8-44e0-8d4f-6582d339e7c6', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Getting quotas for project 3a1f516270354a82995f75c4c149dad2. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Getting quotas for user 592f78cbcf7046e68bcf6c88e039a9be and project 3a1f516270354a82995f75c4c149dad2. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:50:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:50:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5aa92613-b935-434c-8520-6f52713c919c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 1.852432 Apr 12 01:50:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 31/90] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:50:19 2023] POST /compute/v2.1/servers => generated 404 bytes in 1858 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:50:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e5150b79-e3ad-4f63-b9f1-76b1c4add405 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:21 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e5150b79-e3ad-4f63-b9f1-76b1c4add405 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:21 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e5150b79-e3ad-4f63-b9f1-76b1c4add405 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '592f78cbcf7046e68bcf6c88e039a9be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3a1f516270354a82995f75c4c149dad2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e5150b79-e3ad-4f63-b9f1-76b1c4add405 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02" status: 200 len: 1224 microversion: 2.1 time: 0.381603 Apr 12 01:50:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 31/91] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:21 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 => generated 1224 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] Creating event network-changed:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:50:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f3c562a2-9c71-43ce-b074-63fcf5e6e3a4 req-4f26eef5-73df-46af-8307-d3c2a2da2fb5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.217589 Apr 12 01:50:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 30/92] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:21 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] Creating event network-changed:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:50:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c68d0b5c-a098-49d8-8794-0caca78dadb1 req-950836db-6114-4e85-bad0-8c35898b7493 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106930 Apr 12 01:50:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 32/93] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:24 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "name": "network-vif-plugged", "status": "completed", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] Creating event network-vif-plugged:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:50:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-c06862e4-1bd8-4930-9621-c56b6a7a3923 req-b445c10b-1e4d-4e07-a8a5-50dcdae09dc8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091830 Apr 12 01:50:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 32/94] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:27 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] Creating event network-changed:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:50:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-043f42b2-db01-4cda-abd8-8f9105141db9 req-97e8f402-1ad0-459f-9534-fba519613125 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061415 Apr 12 01:50:27 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 31/95] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:27 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "name": "network-vif-plugged", "status": "completed", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] Creating event network-vif-plugged:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:50:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7f3709cd-813e-48a1-9246-11a62f850b79 req-876697ca-be49-47c6-9be3-e1ca1767f513 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033849 Apr 12 01:50:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 33/96] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:27 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "name": "network-vif-plugged", "status": "completed", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}, {"server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "name": "network-vif-plugged", "status": "completed", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:30 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Creating event network-vif-plugged:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:50:30 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] Creating event network-vif-plugged:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:50:30 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e7cb0016-1e11-4577-bd15-67ab5608aa64 req-03e2c9dd-92db-47c2-80b9-66c2df07dec7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.059119 Apr 12 01:50:30 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 33/97] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:29 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:30 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:30 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "name": "network-vif-plugged", "status": "completed", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] Creating event network-vif-plugged:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:50:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-25ee29ba-b5fe-4a5a-885a-4926b13109db req-39f185e9-8df5-473b-9cce-8ab3918368eb service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039396 Apr 12 01:50:30 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 32/98] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:30 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=10 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "name": "network-vif-plugged", "status": "completed", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] Creating event network-vif-plugged:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:50:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-963440eb-a729-4fa0-87dc-fa68e81d30ee req-8457bab2-2e59-4c0b-b710-f59e9057bd07 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042684 Apr 12 01:50:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 34/99] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:32 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=18 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:39 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bacfd73a-5900-41ab-806a-2d428b264442 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1539 microversion: 2.1 time: 0.240863 Apr 12 01:50:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 34/100] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:39 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1539 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:40 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cbaafde1-290b-4a77-b903-bd4b8056464d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 200 len: 1503 microversion: 2.1 time: 0.263721 Apr 12 01:50:40 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 33/101] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:40 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 1503 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:40 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b10c9052-45d9-46b0-af33-add1e2a8c829 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1491 microversion: 2.1 time: 0.307018 Apr 12 01:50:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 35/102] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:40 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1491 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:43 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c503a0141cf4794bf4e6faba59bdba3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a7713a76bb546d2a348927f18b12eb7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-42fca3c1-b9a4-4107-9d72-c6bf064ea652 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 200 len: 1489 microversion: 2.1 time: 0.214430 Apr 12 01:50:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 35/103] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:43 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 1489 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:45 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-9de14af8-bdfe-425d-86b5-63bbe6e8140c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1536 microversion: 2.1 time: 0.286912 Apr 12 01:50:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 34/104] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:50:45 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1536 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:50:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a4a3fb23-7565-49b6-a249-1fe0f515d741 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Action: 'create', calling method: .version_select at 0x7fb203ede170>, body: {"keypair": {"name": "tempest-keypair-1357527151", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBApFLaa/x02EL4eTjTz+moxAIRja1p8gDm3Ncr0KLPMAw5nO769Q6K7EajUuMyNHUQQFEhx8BPuuPQc+tMrIjpiCE+kmukMa/IcXGJ2zZwR2WCPhKNhhV2A/rTHq0zHpIQ=="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:47 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-a4a3fb23-7565-49b6-a249-1fe0f515d741 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Getting quotas for project 3071310920ea42f9a66e1a92d6ecd324. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:47 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-a4a3fb23-7565-49b6-a249-1fe0f515d741 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Getting quotas for user f288eba0f612445596edccf43f08fa22 and project 3071310920ea42f9a66e1a92d6ecd324. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a4a3fb23-7565-49b6-a249-1fe0f515d741 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.32 time: 0.192632 Apr 12 01:50:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 36/105] 10.0.0.210 () {66 vars in 1277 bytes} [Wed Apr 12 01:50:47 2023] POST /compute/v2.1/os-keypairs => generated 401 bytes in 194 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Apr 12 01:50:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-1698777031", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "config_drive": true, "networks": [{"uuid": "c67ab087-9504-4225-ae6f-3584d8223c4c"}, {"port": "1f3154ad-8bf5-46a4-8184-fd6f73576d78", "tag": "port-1"}, {"port": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8", "tag": "port-2"}, {"uuid": "31c0d7f4-5ac9-4f95-840c-237e9eed2f04", "tag": "net-1"}, {"uuid": "31c0d7f4-5ac9-4f95-840c-237e9eed2f04"}, {"uuid": "de0a436a-b743-4519-a522-2468f5ea35a1", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "de0a436a-b743-4519-a522-2468f5ea35a1", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "c7b8c8c2-6939-412f-a1eb-2fdd3cbbd3e8", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "78172d81-0c40-40fc-bbe3-a67e1f56081d", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "3d10e166-577b-475c-a041-3034e9e9e1cb", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--1710454972"}], "key_name": "tempest-keypair-1357527151", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:49 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] validate_networks() for [('c67ab087-9504-4225-ae6f-3584d8223c4c', None, None, None, None, None), (None, None, '1f3154ad-8bf5-46a4-8184-fd6f73576d78', None, None, None), (None, None, 'b7d4528c-dc68-49c5-ad88-00a9ea8689f8', None, None, None), ('31c0d7f4-5ac9-4f95-840c-237e9eed2f04', None, None, None, None, None), ('31c0d7f4-5ac9-4f95-840c-237e9eed2f04', None, None, None, None, None), ('de0a436a-b743-4519-a522-2468f5ea35a1', '10.2.2.100', None, None, None, None), ('de0a436a-b743-4519-a522-2468f5ea35a1', '10.2.2.200', None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:50:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a0bb3ec7-2f44-41b3-980e-8aacc571a94c tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Action: 'create', calling method: .version_select at 0x7fb203ed2050>, body: {"keypair": {"name": "tempest-keypair-1066267509", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFUH4LNxTqx2L62GMVH41PI3IO5q8s9r5Mr8h4YxEzFjxPQNGwWmbswGDWvoP2aOF7udl4Xw+VUjc6jdYHoY2mt1HtgRap0+LKKZX+JXTb8LirKaWn1HwhSxxvs6pK4Zlg=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=16 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:50 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-a0bb3ec7-2f44-41b3-980e-8aacc571a94c tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Getting quotas for project 85c85b2820514000b08e47046f7bd717. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:50 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-a0bb3ec7-2f44-41b3-980e-8aacc571a94c tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Getting quotas for user aa19aa56b6ee4a10a07eb6e421a131bd and project 85c85b2820514000b08e47046f7bd717. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a0bb3ec7-2f44-41b3-980e-8aacc571a94c tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.42 time: 0.067893 Apr 12 01:50:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 35/106] 10.0.0.210 () {66 vars in 1277 bytes} [Wed Apr 12 01:50:50 2023] POST /compute/v2.1/os-keypairs => generated 401 bytes in 70 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Apr 12 01:50:51 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=13 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-1664937775", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "config_drive": true, "networks": [{"uuid": "635d5017-87b6-4086-acb7-464c3f0841ee"}, {"port": "69bce85f-9dda-4202-a608-f9c4da0cff43", "tag": "port-1"}, {"port": "efae59db-bdf2-467e-803b-c2b1a85c10a7", "tag": "port-2"}, {"uuid": "c82cca19-2f7b-4cdd-8fff-a7549f75492e", "tag": "net-1"}, {"uuid": "c82cca19-2f7b-4cdd-8fff-a7549f75492e"}, {"uuid": "2db9de15-fcb7-46ee-97c9-9d16ad878543", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "2db9de15-fcb7-46ee-97c9-9d16ad878543", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "14875542-8ecd-404c-b962-08b834eacb86", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "65104dd2-34df-439c-9659-ca5448f7fda7", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "65567a69-c266-48ba-942d-183231131ad7", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--1545472973"}], "key_name": "tempest-keypair-1066267509", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:51 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=19 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Getting quotas for project 3071310920ea42f9a66e1a92d6ecd324. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Getting quotas for user f288eba0f612445596edccf43f08fa22 and project 3071310920ea42f9a66e1a92d6ecd324. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:51 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] validate_networks() for [('635d5017-87b6-4086-acb7-464c3f0841ee', None, None, None, None, None), (None, None, '69bce85f-9dda-4202-a608-f9c4da0cff43', None, None, None), (None, None, 'efae59db-bdf2-467e-803b-c2b1a85c10a7', None, None, None), ('c82cca19-2f7b-4cdd-8fff-a7549f75492e', None, None, None, None, None), ('c82cca19-2f7b-4cdd-8fff-a7549f75492e', None, None, None, None, None), ('2db9de15-fcb7-46ee-97c9-9d16ad878543', '10.2.2.100', None, None, None, None), ('2db9de15-fcb7-46ee-97c9-9d16ad878543', '10.2.2.200', None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:50:51 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='c7b8c8c2-6939-412f-a1eb-2fdd3cbbd3e8',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='78172d81-0c40-40fc-bbe3-a67e1f56081d',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='3d10e166-577b-475c-a041-3034e9e9e1cb',volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:50:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-c8d857cb-ed37-491c-a549-51dd4134ba55 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.32 time: 2.993533 Apr 12 01:50:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 36/107] 10.0.0.210 () {66 vars in 1266 bytes} [Wed Apr 12 01:50:49 2023] POST /compute/v2.1/servers => generated 404 bytes in 2995 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 01:50:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:52 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-eea8f2a5-289e-4b32-81e7-f92333d647e6 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 200 len: 1280 microversion: 2.32 time: 0.071469 Apr 12 01:50:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 36/108] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:50:52 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 1280 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Getting quotas for project 85c85b2820514000b08e47046f7bd717. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Getting quotas for user aa19aa56b6ee4a10a07eb6e421a131bd and project 85c85b2820514000b08e47046f7bd717. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:50:53 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='14875542-8ecd-404c-b962-08b834eacb86',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='65104dd2-34df-439c-9659-ca5448f7fda7',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='65567a69-c266-48ba-942d-183231131ad7',volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:50:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5105616e-60ff-4963-bf48-503d3b22ddc7 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.42 time: 2.713245 Apr 12 01:50:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 37/109] 10.0.0.210 () {66 vars in 1266 bytes} [Wed Apr 12 01:50:51 2023] POST /compute/v2.1/servers => generated 404 bytes in 2715 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 01:50:54 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:50:54 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:50:54 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:54 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:54 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:50:54 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-92a1cbb9-1b11-4d57-8742-c8dd8c0cc0c6 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 200 len: 1280 microversion: 2.42 time: 0.077441 Apr 12 01:50:54 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 37/110] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:50:54 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 1280 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:50:58 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=19 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:50:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] Creating event network-changed:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:50:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-96048dec-c1cc-42a0-89bf-f4ef297bf2af req-e715006c-98a6-4216-8fdb-cb48d0ab0d32 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034202 Apr 12 01:50:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 37/111] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:58 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:50:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:50:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:50:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:50:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:50:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] Creating event network-changed:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:50:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8a462bc3-ab62-45c7-8d4e-f4be44320870 req-39a78ad7-23d2-42c7-a05f-0f2233ed8364 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034713 Apr 12 01:50:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 38/112] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:50:59 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "1f3154ad-8bf5-46a4-8184-fd6f73576d78"}, {"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Creating event network-changed:1f3154ad-8bf5-46a4-8184-fd6f73576d78 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] Creating event network-changed:b7d4528c-dc68-49c5-ad88-00a9ea8689f8 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-854dd1cc-52a8-4194-a629-7d07532e7de6 req-07218f17-8dbd-4a54-a379-c6924c8a43f9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.037565 Apr 12 01:51:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 38/113] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:00 2023] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:01 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "69bce85f-9dda-4202-a608-f9c4da0cff43"}, {"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "efae59db-bdf2-467e-803b-c2b1a85c10a7"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Creating event network-changed:69bce85f-9dda-4202-a608-f9c4da0cff43 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] Creating event network-changed:efae59db-bdf2-467e-803b-c2b1a85c10a7 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-d3636de9-58cf-4943-b719-20c89e25dc7b req-d848722a-e67f-4e91-b1e1-f94a81ecf8d9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.033839 Apr 12 01:51:01 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 38/114] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:01 2023] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}, {"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}, {"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Creating event network-changed:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Creating event network-changed:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] Creating event network-changed:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-5e14ffa0-9c28-4cb9-ae40-7456fda86c12 req-989f4362-6e00-4524-8ea9-fcfdbeedbcc5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.037251 Apr 12 01:51:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 39/115] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:02 2023] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}, {"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:03 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Creating event network-changed:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] Creating event network-changed:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-6e2cf5ef-f0bc-46cf-b71f-ec418947b212 req-c1ac70ed-5d3a-49db-b7e8-f7444beb509c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.040640 Apr 12 01:51:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 39/116] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:03 2023] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:04 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}, {"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}, {"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:04 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Creating event network-changed:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Creating event network-changed:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] Creating event network-changed:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-3d2a2573-dda2-4919-8645-0cfb96419ee2 req-550b7933-8e50-4c65-b9b0-1e223c5e0479 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.035067 Apr 12 01:51:04 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 39/117] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:04 2023] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=10 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:04 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e400a3359c844c449445cfba08ef7a7b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '742d0477bb3b464e8d98d42a85c330e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:04 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4023304a-29b5-46f2-9d63-76603f74910a tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 200 len: 1547 microversion: 2.1 time: 0.214670 Apr 12 01:51:04 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 40/118] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:04 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 1547 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:09 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-92201a0b-23ab-4f35-961b-240c5861f677 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335" status: 200 len: 1544 microversion: 2.1 time: 0.169230 Apr 12 01:51:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 40/119] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:09 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 => generated 1544 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:11 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '592f78cbcf7046e68bcf6c88e039a9be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3a1f516270354a82995f75c4c149dad2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-246967f6-1ff6-476c-8ec8-587d5b6ac755 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02" status: 200 len: 1524 microversion: 2.1 time: 0.156013 Apr 12 01:51:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 40/120] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:11 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 => generated 1524 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=20,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=12 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] Creating event network-vif-plugged:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f29f002c-e8e3-4f5d-ab9c-e0c491e19d58 req-39da3c59-3c94-4676-8d37-87f7f7012b87 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043471 Apr 12 01:51:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 41/121] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:28 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "69bce85f-9dda-4202-a608-f9c4da0cff43"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=11 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] Creating event network-vif-plugged:69bce85f-9dda-4202-a608-f9c4da0cff43 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-9a660ffb-2943-4a4c-8ade-f54aba5ec167 req-179b14e6-9f1b-4475-8282-825a11713c45 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060731 Apr 12 01:51:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 41/122] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:29 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "efae59db-bdf2-467e-803b-c2b1a85c10a7"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=22,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] Creating event network-vif-plugged:efae59db-bdf2-467e-803b-c2b1a85c10a7 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-6f5b65aa-0874-4305-ac4e-292af80371d5 req-9bd85008-12c8-4843-a0ae-31b4bde66d60 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060637 Apr 12 01:51:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 41/123] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:29 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] Creating event network-vif-plugged:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-80d64e34-c1d8-4952-8f65-53a7dc22f3ad req-31686057-6cd8-4fc6-b512-11195b9c451e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030240 Apr 12 01:51:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 42/124] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:30 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "69bce85f-9dda-4202-a608-f9c4da0cff43"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Creating event network-vif-plugged:69bce85f-9dda-4202-a608-f9c4da0cff43 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Creating event network-vif-plugged:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] Creating event network-vif-plugged:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8fb1c2bd-a035-4f08-af76-179cdaee1e00 req-382a8477-5e90-4f3a-a165-445b96546ff5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.040807 Apr 12 01:51:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 42/125] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:31 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:31 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "efae59db-bdf2-467e-803b-c2b1a85c10a7"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Creating event network-vif-plugged:efae59db-bdf2-467e-803b-c2b1a85c10a7 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Creating event network-vif-plugged:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] Creating event network-vif-plugged:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-2e32e494-39ce-43cf-9afa-f9df39f29d44 req-ecd59969-200f-4887-87a0-f5fe5e5fd687 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.041770 Apr 12 01:51:31 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 42/126] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:31 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Creating event network-vif-plugged:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Creating event network-vif-plugged:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Creating event network-vif-plugged:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] Creating event network-vif-plugged:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3682ebaf-0fa3-4efc-98c1-23fb0ab80258 req-4cb49a14-944b-4939-97c1-2718b2559b4a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 696 microversion: 2.1 time: 0.039324 Apr 12 01:51:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 43/127] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:32 2023] POST /compute/v2.1/os-server-external-events => generated 696 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "1f3154ad-8bf5-46a4-8184-fd6f73576d78"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "1f3154ad-8bf5-46a4-8184-fd6f73576d78"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:1f3154ad-8bf5-46a4-8184-fd6f73576d78 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:1f3154ad-8bf5-46a4-8184-fd6f73576d78 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] Creating event network-vif-plugged:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8e146797-3f8d-4efd-86ff-b1e370109459 req-b84d4278-d385-4c71-a93f-a6367590ea07 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 1038 microversion: 2.1 time: 0.035521 Apr 12 01:51:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 43/128] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:33 2023] POST /compute/v2.1/os-server-external-events => generated 1038 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] Creating event network-vif-plugged:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-df8f08f0-8676-4f76-aaf4-3d1164249013 req-7966a1e8-0ff1-4dfb-9e47-2050cb72b88a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051080 Apr 12 01:51:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 43/129] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:33 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Creating event network-vif-plugged:b7d4528c-dc68-49c5-ad88-00a9ea8689f8 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] Creating event network-vif-plugged:b7d4528c-dc68-49c5-ad88-00a9ea8689f8 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-11536a1c-caff-4687-9b97-b69839fdcbd9 req-93f2d1b3-8f8d-4a45-aaee-cd916faf9bb0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.030754 Apr 12 01:51:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 44/130] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:34 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Creating event network-vif-plugged:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] Creating event network-vif-plugged:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3b5a7794-1aa1-4b7f-969f-757f54670d44 req-c46deae6-0621-431e-afd6-016f64a894b6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.028960 Apr 12 01:51:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 44/131] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:35 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Creating event network-vif-plugged:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Creating event network-vif-plugged:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] Creating event network-vif-plugged:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:51:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-aed2017f-da1b-4f11-b9f1-8202d8cce215 req-c23ae1b4-3ef9-49d7-ae18-92e1bf6c1873 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.031422 Apr 12 01:51:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 44/132] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:35 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c0a31b1e-b186-4ee2-99c6-1c4a391855f1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.170024 Apr 12 01:51:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 45/133] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:51:39 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-interface => generated 283 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:40 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lazy-loading 'flavor' on Instance uuid d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:51:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-96121c63-7660-4561-aa63-984cc88f0361 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9/action" status: 200 len: 32 microversion: 2.1 time: 0.430189 Apr 12 01:51:40 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 45/134] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:51:40 2023] POST /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9/action => generated 32 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1705395768", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] validate_networks() for [('1d227a8b-1ad3-4bc6-a32e-4322c8f75eba', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-09e66ac5-0643-459e-beec-98c7a986a9b9 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 200 len: 1509 microversion: 2.1 time: 0.405812 Apr 12 01:51:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 45/135] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:40 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 1509 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] Creating event network-changed:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:51:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8c595e5e-9df8-4361-93c0-a78129f83c93 req-908b7efd-0b40-4310-baf2-2603d2412333 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037197 Apr 12 01:51:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 46/136] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:41 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:41 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-588098d0-32c3-4092-9226-0eb9b432ab46 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1539 microversion: 2.1 time: 0.182334 Apr 12 01:51:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 46/137] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:41 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1539 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Getting quotas for project 021f9aee2b7849ef8fcd67d6d6522d27. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Getting quotas for user 728c534ef1f54c50ad2b619b44c3b9c7 and project 021f9aee2b7849ef8fcd67d6d6522d27. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:51:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-322b2e37-c7be-4630-b267-06577668094e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 1.093210 Apr 12 01:51:42 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 46/138] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:51:41 2023] POST /compute/v2.1/servers => generated 378 bytes in 1096 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:51:42 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bf15e041-11e3-4c27-9a96-3c8137f0aa98 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:42 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bf15e041-11e3-4c27-9a96-3c8137f0aa98 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:42 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-bf15e041-11e3-4c27-9a96-3c8137f0aa98 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:42 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bf15e041-11e3-4c27-9a96-3c8137f0aa98 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 200 len: 1211 microversion: 2.1 time: 0.084500 Apr 12 01:51:42 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 47/139] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:42 2023] GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 1211 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:42 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-79f4e47f-a0c2-4021-97d1-f13667f66984 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1539 microversion: 2.1 time: 0.417392 Apr 12 01:51:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 47/140] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:42 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1539 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:42 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-23ebecd1-3f00-4ee4-8715-91b64b1e9ff2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 200 len: 2825 microversion: 2.32 time: 0.417239 Apr 12 01:51:42 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 47/141] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:51:42 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 2825 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:51:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-unplugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}, {"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Creating event network-vif-unplugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:51:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:51:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c59a51e8-f4c3-4cd3-b4ec-2e72bff3eece req-e47db366-4c49-4f4b-9ae8-34313b5c9f3a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.059293 Apr 12 01:51:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 48/142] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:43 2023] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:44 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-16aac15a-ee56-4b71-ab1d-964c7b027077 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 200 len: 2829 microversion: 2.42 time: 0.289340 Apr 12 01:51:44 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 48/143] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:51:44 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 2829 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b9fb6599-c30a-4a61-ab9b-ae62affb4857"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}, {"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:51:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 01:51:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-56859869-d555-4518-a3e1-c33c3989f1a1 req-a3cd267b-048a-4f37-88a2-dbc1a8238637 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.030614 Apr 12 01:51:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 49/144] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:45 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5d1aa4dc-4d27-4118-ac09-46e361f6dafc tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.093530 Apr 12 01:51:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 49/145] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:51:45 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-interface => generated 282 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] Creating event network-changed:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:51:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-17ad661f-2d0d-45f5-94ee-f7b637a8d24d req-490e7fc1-394a-49c0-a14d-555eda23b201 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030567 Apr 12 01:51:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 50/146] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:47 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:47 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d7a05f1c-64de-4d5c-b3b9-bd9d6427ea9f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/ips" status: 200 len: 104 microversion: 2.1 time: 0.039632 Apr 12 01:51:47 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 50/147] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 01:51:47 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/ips => generated 104 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4a1b5cab-304f-446e-a918-d70b6dd613a9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "POST /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.655759 Apr 12 01:51:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 48/148] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:51:45 2023] POST /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c/os-volume_attachments => generated 194 bytes in 3660 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "52725f14-a644-4019-b7c6-9547d83cfb1b"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] Creating event network-changed:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:51:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-a458b47c-eea5-4267-b24e-6bba9c2fed2b req-74daab3a-6015-4f9d-b778-53b4b57e24db service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047899 Apr 12 01:51:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 51/149] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:49 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-81d76f90-d832-4023-b3b1-8e31cd0c71de tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.624756 Apr 12 01:51:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 51/150] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:51:48 2023] POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments => generated 194 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "name": "network-vif-plugged", "status": "completed", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] Creating event network-vif-plugged:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:51:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-ab30c288-6949-4cf7-b0de-cbaa6d6c2290 req-554a937b-c291-44af-9bab-1cc21f5039d7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044804 Apr 12 01:51:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 49/151] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:51 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "name": "network-vif-plugged", "status": "completed", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] Creating event network-vif-plugged:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:51:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-698aada5-734d-45a9-962d-72c54eb0fd12 req-2a37b225-c566-4ea5-818c-4cbe23c5ce68 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031893 Apr 12 01:51:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 52/152] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:57 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:51:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-566d5dcd-1a35-45f5-b562-a9e4f15624d9 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 204 len: 0 microversion: 2.1 time: 0.369430 Apr 12 01:51:57 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 52/153] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:51:57 2023] DELETE /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:57 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6c503a0141cf4794bf4e6faba59bdba3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a7713a76bb546d2a348927f18b12eb7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "name": "network-vif-unplugged", "status": "completed", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-195701a2-f134-47ce-91a7-34faee7e3c04 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 200 len: 1541 microversion: 2.1 time: 0.288938 Apr 12 01:51:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 50/154] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:57 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 1541 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:51:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] Creating event network-vif-unplugged:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:51:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-c6863eb5-1fd7-4118-bdad-250a876444e9 req-680388fc-b58e-4ccb-a9f6-759322ae28c9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030664 Apr 12 01:51:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 53/155] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:51:57 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:51:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:51:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:58 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Going to try to stop instance {{(pid=72506) force_stop /opt/stack/nova/nova/compute/api.py:2764}} Apr 12 01:51:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-39d8cb83-0db3-4a02-a9ac-4da96c12cdd2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action" status: 202 len: 0 microversion: 2.1 time: 0.128391 Apr 12 01:51:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 53/156] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:51:58 2023] POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:51:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:51:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-49272f7d-0914-4a7b-9726-921d34f7b3a5 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1711 microversion: 2.1 time: 0.198714 Apr 12 01:51:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 51/157] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:51:58 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1711 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "name": "network-vif-deleted", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "371f55da-306a-4ef9-b04f-7c22e8071e5c", "name": "network-vif-plugged", "status": "completed", "tag": "f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1"}, {"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-unplugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}, {"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] Creating event network-vif-deleted:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:52:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e297c1f7-1423-4b5e-942f-229e5436706c req-cd7c2fc2-903e-4466-b527-15ac477a6f6f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039809 Apr 12 01:52:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 54/158] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:00 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Creating event network-vif-plugged:f70a5cfd-b9e1-4585-bb0c-5c8f0f3bb1f1 for instance 371f55da-306a-4ef9-b04f-7c22e8071e5c on user Apr 12 01:52:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Creating event network-vif-unplugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:52:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:52:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a0451880-0039-46b1-b5af-e9f0abdb0718 req-93b65782-f143-46ad-8155-8245bacae58e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 527 microversion: 2.1 time: 0.046229 Apr 12 01:52:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 54/159] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:00 2023] POST /compute/v2.1/os-server-external-events => generated 527 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-21852fa1-89ba-432e-b690-e129ea46f436 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.092906 Apr 12 01:52:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 52/160] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:52:04 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-interface => generated 282 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=29 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:05 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:05 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] Creating event network-changed:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:52:05 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-f4898349-6c1c-4c00-9cdc-61223681d895 req-e56407e8-a764-43f6-8eba-cb70d9abaf53 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045202 Apr 12 01:52:05 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 55/161] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:05 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "616face5-aa52-40cc-98f1-87cecb1e9bef"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-ae649a75-9990-4f10-9080-fe8e5d9e2250 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "POST /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.334943 Apr 12 01:52:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 55/162] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:52:07 2023] POST /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments => generated 194 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-11d456b3-9bb3-4449-a211-cdb5ba5eb3e1 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.085397 Apr 12 01:52:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 53/163] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:52:09 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-interface => generated 282 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] Creating event network-changed:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:52:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-c6fe0d8a-275a-458d-be52-2dfc76db767a req-c89000f1-ad96-41c4-8110-63192cc3ccd1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034542 Apr 12 01:52:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 56/164] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:10 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-df795f8d-5fb9-4a6a-b481-0353066f88ef tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.088011 Apr 12 01:52:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 56/165] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:52:11 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-interface => generated 283 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d6ab4b69-8603-4354-b1d4-9fd31c07913a tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "POST /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.361876 Apr 12 01:52:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 54/166] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:52:12 2023] POST /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments => generated 194 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "30e993bc-91f0-45c4-b5fa-5781522df5fb"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:12 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:12 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] Creating event network-changed:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:52:12 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a1f26554-1bda-404e-8d9d-4c26ef07e2fe req-67957465-9ab7-4c4e-afd6-0c453ac288aa service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029247 Apr 12 01:52:12 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 57/167] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:12 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-56081dfc-ea07-4189-8a7e-2922d194cb58 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "POST /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.285498 Apr 12 01:52:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 57/168] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:52:12 2023] POST /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 194 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7a48aab5-3d93-4e5a-9bc4-91aac7671eb4 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049132 Apr 12 01:52:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 55/169] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:14 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:14 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lazy-loading 'services' on Instance uuid f2eedf78-0fd6-4cc1-8efd-50958e07fde6 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:52:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-45c0674a-5bed-463d-aa4d-76cd607afba3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments/616face5-aa52-40cc-98f1-87cecb1e9bef" status: 202 len: 0 microversion: 2.1 time: 0.238379 Apr 12 01:52:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 58/170] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:52:14 2023] DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments/616face5-aa52-40cc-98f1-87cecb1e9bef => generated 0 bytes in 240 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-9cbff1c7-d838-4d6a-9531-d82448708eaf tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.062825 Apr 12 01:52:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 58/171] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:16 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments => generated 25 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8ef1fe0a-634e-439d-be3c-b667cc38b0ac tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.040321 Apr 12 01:52:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 56/172] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:16 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6/os-volume_attachments => generated 25 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:52:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bcba319d-011e-461a-8cb4-2a10fad8d418 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 204 len: 0 microversion: 2.1 time: 0.112023 Apr 12 01:52:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 59/173] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:52:16 2023] DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e400a3359c844c449445cfba08ef7a7b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '742d0477bb3b464e8d98d42a85c330e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "name": "network-vif-unplugged", "status": "completed", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0a4225fb-3025-47dd-a0b6-f8de42006fb7 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 200 len: 1670 microversion: 2.1 time: 0.250275 Apr 12 01:52:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 59/174] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:16 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 1670 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] Creating event network-vif-unplugged:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:52:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-82d5de8d-43fc-4eab-9d21-98efb9767b80 req-d0e49049-d33c-4e14-8f54-f9988a53c505 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034067 Apr 12 01:52:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 57/175] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:16 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "name": "network-vif-plugged", "status": "completed", "tag": "38b808a5-094e-480a-b52f-d36407148227"}, {"server_uuid": "f2eedf78-0fd6-4cc1-8efd-50958e07fde6", "name": "network-vif-deleted", "tag": "38b808a5-094e-480a-b52f-d36407148227"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Creating event network-vif-plugged:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:52:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] Creating event network-vif-deleted:38b808a5-094e-480a-b52f-d36407148227 for instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 on user Apr 12 01:52:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f0edfeee-e89e-4103-ae6b-91e56bc7f55c req-0afc50c9-e25d-479c-9852-0aef984f9c85 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.047690 Apr 12 01:52:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 60/176] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:18 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "bf7af630-d5dc-4b53-ae75-d25ff9e0564e"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] HTTP exception thrown: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Request method failure captured: Apr 12 01:52:19 user devstack@n-api.service[72507]: request: PUT /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments/d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a HTTP/1.1 Apr 12 01:52:19 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:52:19 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:52:19 user devstack@n-api.service[72507]: Connection: close Apr 12 01:52:19 user devstack@n-api.service[72507]: Content-Length: 74 Apr 12 01:52:19 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:52:19 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:52:19 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:52:19 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg5UvNwlL2HfJvouJSPFlA70KbFzpbR54yZdOYTf2O4RCztQ4EXaY_isxAYv6gRvvlB5VnkDPHcBwobbudzUTyZ2wOM2sc9NfXUE52BhBoKIu53heAEM92sJrqpIDjSuDcYrWMJidDfpkZjZsznwXMHxwhSyg6Rt8mVHD9a9RSIC_5M Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Project-Id: 37f0f8a76a6a4ec6a9caa26de4805b6f Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Role: member,reader,admin Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Roles: member,reader,admin Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Tenant-Id: 37f0f8a76a6a4ec6a9caa26de4805b6f Apr 12 01:52:19 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:52:19 user devstack@n-api.service[72507]: X-User: tempest-VolumesAdminNegativeTest-807677884-project-admin Apr 12 01:52:19 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:52:19 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:52:19 user devstack@n-api.service[72507]: X-User-Id: 4c7379a5c55645669ac6cd63053f58e8 Apr 12 01:52:19 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesAdminNegativeTest-807677884-project-admin Apr 12 01:52:19 user devstack@n-api.service[72507]: Apr 12 01:52:19 user devstack@n-api.service[72507]: {"volumeAttachment": {"volumeId": "bf7af630-d5dc-4b53-ae75-d25ff9e0564e"}} Apr 12 01:52:19 user devstack@n-api.service[72507]: method: > Apr 12 01:52:19 user devstack@n-api.service[72507]: exception: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. Apr 12 01:52:19 user devstack@n-api.service[72507]: action_args: {'server_id': '2dbe5433-e1e5-4b85-a0f7-904c3b784335', 'id': 'd5ebfb2e-aa8c-4745-b23a-09c29a6dc37a', 'body': {'volumeAttachment': {'volumeId': 'bf7af630-d5dc-4b53-ae75-d25ff9e0564e'}}} Apr 12 01:52:19 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. (HTTP 404) (Request-ID: req-a12c5ea3-0da6-4823-bcf1-ac50b4531035) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 436, in _update_volume_swap Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi new_volume = self.volume_api.get(context, new_volume_id) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 445, in _update_volume_swap Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. Apr 12 01:52:19 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Returning 400 to user: Volume bf7af630-d5dc-4b53-ae75-d25ff9e0564e could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:52:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5ce51751-8f13-4007-b782-02a765950589 tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] 10.0.0.210 "PUT /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments/d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a" status: 400 len: 107 microversion: 2.1 time: 0.315491 Apr 12 01:52:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 60/177] 10.0.0.210 () {64 vars in 1510 bytes} [Wed Apr 12 01:52:19 2023] PUT /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments/d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a => generated 107 bytes in 320 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:19 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lazy-loading 'services' on Instance uuid 2dbe5433-e1e5-4b85-a0f7-904c3b784335 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:52:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-87686348-ee65-4e81-90b8-5b25a8b3e7d4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments/d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a" status: 202 len: 0 microversion: 2.1 time: 0.237404 Apr 12 01:52:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 58/178] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:52:19 2023] DELETE /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments/d5ebfb2e-aa8c-4745-b23a-09c29a6dc37a => generated 0 bytes in 239 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:20 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b6aa09b7-a1a2-4c44-ba33-694a9f0aa6c4 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047807 Apr 12 01:52:20 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 61/179] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:20 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "d08e265d-55d4-4542-a7fe-359cd0740624"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a76f9616-5e21-4c1c-bb56-56b06c96ea11 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] 10.0.0.210 "PUT /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/30e993bc-91f0-45c4-b5fa-5781522df5fb" status: 202 len: 0 microversion: 2.1 time: 0.464235 Apr 12 01:52:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 61/180] 10.0.0.210 () {64 vars in 1510 bytes} [Wed Apr 12 01:52:20 2023] PUT /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/30e993bc-91f0-45c4-b5fa-5781522df5fb => generated 0 bytes in 466 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=13 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=41,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:31 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9af0080a-3e48-4ccd-9a84-9ce2bf68c54a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 200 len: 1489 microversion: 2.1 time: 0.172877 Apr 12 01:52:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 59/181] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:31 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 1489 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=13 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=39,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-ac5379f5-7f13-4ae5-9b31-c8fff86da73d tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 200 len: 1493 microversion: 2.1 time: 0.177074 Apr 12 01:52:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 62/182] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:32 2023] GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 1493 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-692b6f1b-39dc-4643-ad95-481ffeff1a07 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1657 microversion: 2.1 time: 0.193977 Apr 12 01:52:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 62/183] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:32 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1657 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-35a26390-8ce4-4a57-ba0d-150b297cdc7d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1657 microversion: 2.1 time: 0.156267 Apr 12 01:52:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 60/184] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:32 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1657 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b2bf2833-ca63-42ea-81a3-0732094fe73c tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040332 Apr 12 01:52:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 63/185] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:33 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1195ef3e-5a5a-453b-857c-18403b1b8ac2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041892 Apr 12 01:52:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 63/186] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:33 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "30e993bc-91f0-45c4-b5fa-5781522df5fb"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-190d0778-f1a6-4127-bd6b-026451758b43 tempest-TestVolumeSwap-1872168700 tempest-TestVolumeSwap-1872168700-project-admin] 10.0.0.210 "PUT /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/d08e265d-55d4-4542-a7fe-359cd0740624" status: 202 len: 0 microversion: 2.1 time: 0.302558 Apr 12 01:52:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 61/187] 10.0.0.210 () {64 vars in 1510 bytes} [Wed Apr 12 01:52:33 2023] PUT /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/d08e265d-55d4-4542-a7fe-359cd0740624 => generated 0 bytes in 304 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-60198970-f6a8-4165-96b4-d3a4e13eeae2 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7/os-interface" status: 200 len: 1829 microversion: 2.32 time: 0.147623 Apr 12 01:52:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 64/188] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 01:52:42 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7/os-interface => generated 1829 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] Creating event network-changed:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-f3ffb7f0-a37b-491c-9382-51ae496eb949 req-c23d2932-51f5-4a6a-baab-ebef74cfe9d4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044498 Apr 12 01:52:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 64/189] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:43 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:44 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:44 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:44 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:44 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3f3596b1-55b9-4c61-97a9-bc8ff2a2c506 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 200 len: 2825 microversion: 2.32 time: 0.252947 Apr 12 01:52:44 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 62/190] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:52:43 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 2825 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-11f4bbd9-fb30-43fe-b180-464962d50308 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7/os-interface" status: 200 len: 1829 microversion: 2.32 time: 0.123261 Apr 12 01:52:44 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 65/191] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 01:52:44 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7/os-interface => generated 1829 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d6f40edd-4a36-4e0b-aa04-eae80b4c19cf tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2/os-interface" status: 200 len: 1829 microversion: 2.42 time: 0.137551 Apr 12 01:52:44 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 65/192] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 01:52:44 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2/os-interface => generated 1829 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-92275ae0-ff32-4c64-b1b7-5a7aa46aa27b tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052751 Apr 12 01:52:45 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 63/193] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:45 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-39fe4c44-8a48-4bf0-8b1e-f624e85c5433 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.040790 Apr 12 01:52:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 66/194] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:45 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 197 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lazy-loading 'services' on Instance uuid 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:52:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-be6b807c-2ce6-4b03-9103-20ed027f62de tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/30e993bc-91f0-45c4-b5fa-5781522df5fb" status: 202 len: 0 microversion: 2.1 time: 0.252563 Apr 12 01:52:45 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 66/195] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:52:45 2023] DELETE /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments/30e993bc-91f0-45c4-b5fa-5781522df5fb => generated 0 bytes in 254 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9b00bf32-a814-404b-9ee1-1e5e326d1ed5 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 204 len: 0 microversion: 2.32 time: 0.226211 Apr 12 01:52:45 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 64/196] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:52:45 2023] DELETE /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d65bc0ef-b1f9-475c-b4dd-4e3903ea3da2 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051390 Apr 12 01:52:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 67/197] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:45 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:45 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] Creating event network-changed:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:45 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4502ddd0-54a6-49fe-afc0-dd46e12c5d59 req-3bb7844c-ed2c-4b46-b45c-c7cc1b6ff16c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039100 Apr 12 01:52:45 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 68/198] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:45 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:45 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6c3ea3f5-3005-4fbc-926d-8794ff1c924b tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 200 len: 2829 microversion: 2.42 time: 0.251979 Apr 12 01:52:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 67/199] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:52:45 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 2829 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f288eba0f612445596edccf43f08fa22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3071310920ea42f9a66e1a92d6ecd324', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-81780fb2-63b4-4a41-87af-f6c20371177d tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 200 len: 2950 microversion: 2.32 time: 0.260317 Apr 12 01:52:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 65/200] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:52:45 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 2950 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-630a185d-a3c7-4f4a-8305-7a9786ff260d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2/os-interface" status: 200 len: 1829 microversion: 2.42 time: 0.195443 Apr 12 01:52:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 69/201] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 01:52:46 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2/os-interface => generated 1829 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "1f3154ad-8bf5-46a4-8184-fd6f73576d78"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] Creating event network-vif-unplugged:1f3154ad-8bf5-46a4-8184-fd6f73576d78 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-1e64ad82-334f-40e3-b6a4-9ae03f08b5c1 req-001ddbcf-6554-4e6a-8ed3-be381658909f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033820 Apr 12 01:52:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 68/202] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:46 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] Creating event network-vif-unplugged:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-5903ddca-4625-4df5-b6c1-9c86717bf16f req-3f45a993-d462-40d9-ab6a-0c506b289ea4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039447 Apr 12 01:52:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 66/203] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:46 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:52:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4513b53b-77e7-4636-904f-8e1221a8adf8 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 204 len: 0 microversion: 2.42 time: 0.182590 Apr 12 01:52:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 70/204] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:52:47 2023] DELETE /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b7d4528c-dc68-49c5-ad88-00a9ea8689f8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Creating event network-vif-unplugged:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Creating event network-vif-plugged:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Creating event network-vif-unplugged:b7d4528c-dc68-49c5-ad88-00a9ea8689f8 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] Creating event network-vif-plugged:b7d4528c-dc68-49c5-ad88-00a9ea8689f8 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3c162394-817b-4b55-842c-791e13060922 req-cada203e-d7f2-42be-b159-6f30d5eb4cda service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 700 microversion: 2.1 time: 0.036586 Apr 12 01:52:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 67/205] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:47 2023] POST /compute/v2.1/os-server-external-events => generated 700 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa19aa56b6ee4a10a07eb6e421a131bd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '85c85b2820514000b08e47046f7bd717', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f9f80532-a27c-423a-b961-b37a4cc77d36 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 200 len: 2953 microversion: 2.42 time: 0.263551 Apr 12 01:52:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 69/206] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:52:47 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 2953 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] HTTP exception thrown: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Request method failure captured: Apr 12 01:52:48 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c HTTP/1.1 Apr 12 01:52:48 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:52:48 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:52:48 user devstack@n-api.service[72506]: Connection: close Apr 12 01:52:48 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:52:48 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:52:48 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:52:48 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:52:48 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg4-2IuAvY8X27x8fuRHC5yFoZYmF2F5nMvBeQijMVRA7BNZAqo8zfRB4YXRg8Rtn9woRGnsVENU1mbuvazrfbB7Z1ZAJ_rKRBtjyh5du3wB1IpKPxEs08IK0vyR4rdpr1qbvSqL1t2-b2jHSMeAU-Dl0i2maVbglwr7QeaAhEWjZFo Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Project-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Project-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Tenant: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Tenant-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-User: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:48 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:52:48 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:52:48 user devstack@n-api.service[72506]: X-User-Id: 6c503a0141cf4794bf4e6faba59bdba3 Apr 12 01:52:48 user devstack@n-api.service[72506]: X-User-Name: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:48 user devstack@n-api.service[72506]: method: > Apr 12 01:52:48 user devstack@n-api.service[72506]: exception: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72506]: action_args: {'id': '371f55da-306a-4ef9-b04f-7c22e8071e5c'} Apr 12 01:52:48 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Returning 404 to user: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:52:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a7de2d25-c18c-4e4f-ab0e-54355996db7c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 404 len: 111 microversion: 2.1 time: 0.037801 Apr 12 01:52:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 71/207] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:48 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] HTTP exception thrown: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Request method failure captured: Apr 12 01:52:48 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c/os-volume_attachments HTTP/1.1 Apr 12 01:52:48 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:52:48 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:52:48 user devstack@n-api.service[72505]: Connection: close Apr 12 01:52:48 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:52:48 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:52:48 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:52:48 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:52:48 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg4-2IuAvY8X27x8fuRHC5yFoZYmF2F5nMvBeQijMVRA7BNZAqo8zfRB4YXRg8Rtn9woRGnsVENU1mbuvazrfbB7Z1ZAJ_rKRBtjyh5du3wB1IpKPxEs08IK0vyR4rdpr1qbvSqL1t2-b2jHSMeAU-Dl0i2maVbglwr7QeaAhEWjZFo Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Project-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Project-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Tenant: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Tenant-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-User: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:48 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:52:48 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:52:48 user devstack@n-api.service[72505]: X-User-Id: 6c503a0141cf4794bf4e6faba59bdba3 Apr 12 01:52:48 user devstack@n-api.service[72505]: X-User-Name: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:48 user devstack@n-api.service[72505]: method: > Apr 12 01:52:48 user devstack@n-api.service[72505]: exception: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72505]: action_args: {'server_id': '371f55da-306a-4ef9-b04f-7c22e8071e5c'} Apr 12 01:52:48 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Returning 404 to user: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9ce024f5-e851-432f-a087-8a6a04f3bd0c tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.054923 Apr 12 01:52:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 68/208] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:52:48 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c/os-volume_attachments => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "1f3154ad-8bf5-46a4-8184-fd6f73576d78"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "69bce85f-9dda-4202-a608-f9c4da0cff43"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-plugged:1f3154ad-8bf5-46a4-8184-fd6f73576d78 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-unplugged:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-plugged:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-unplugged:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-plugged:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] Creating event network-vif-unplugged:69bce85f-9dda-4202-a608-f9c4da0cff43 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-95017d2c-dcf8-4a9c-8ce7-056e0a640dfe req-8e395a21-1a39-4cb7-8397-914034b6ee5e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 1044 microversion: 2.1 time: 0.049216 Apr 12 01:52:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 70/209] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:48 2023] POST /compute/v2.1/os-server-external-events => generated 1044 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-plugged", "status": "completed", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "efae59db-bdf2-467e-803b-c2b1a85c10a7"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "efae59db-bdf2-467e-803b-c2b1a85c10a7"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Creating event network-vif-plugged:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Creating event network-vif-unplugged:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Creating event network-vif-plugged:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Creating event network-vif-unplugged:efae59db-bdf2-467e-803b-c2b1a85c10a7 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] Creating event network-vif-plugged:efae59db-bdf2-467e-803b-c2b1a85c10a7 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3fb14a05-5972-4926-b1db-97b6e3f1e1ef req-072ac15f-e45c-44f4-a7bc-bce2e904e4ae service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 871 microversion: 2.1 time: 0.051508 Apr 12 01:52:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 69/210] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:48 2023] POST /compute/v2.1/os-server-external-events => generated 871 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:48 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:52:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-1e5002d6-f018-4456-b699-57f3e7778e0a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1688 microversion: 2.1 time: 0.338734 Apr 12 01:52:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 72/211] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:48 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1688 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:52:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Creating event network-vif-unplugged:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Creating event network-vif-plugged:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Creating event network-vif-unplugged:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] Creating event network-vif-plugged:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8aeb34a7-adb6-4755-aa37-f6e1eefa3342 req-4d666761-b766-482a-a185-9e385ec26a69 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 700 microversion: 2.1 time: 0.034221 Apr 12 01:52:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 71/212] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:49 2023] POST /compute/v2.1/os-server-external-events => generated 700 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "69bce85f-9dda-4202-a608-f9c4da0cff43"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Creating event network-vif-plugged:69bce85f-9dda-4202-a608-f9c4da0cff43 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Creating event network-vif-unplugged:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Creating event network-vif-plugged:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Creating event network-vif-unplugged:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] Creating event network-vif-plugged:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-da532fdc-5b36-4ecc-9fcf-de24ae2e7f75 req-d86225c2-ab83-40e4-9885-56c63e4220d9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 871 microversion: 2.1 time: 0.049702 Apr 12 01:52:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 70/213] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:50 2023] POST /compute/v2.1/os-server-external-events => generated 871 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] HTTP exception thrown: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Request method failure captured: Apr 12 01:52:50 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c HTTP/1.1 Apr 12 01:52:50 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:52:50 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:52:50 user devstack@n-api.service[72506]: Connection: close Apr 12 01:52:50 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:52:50 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:52:50 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:52:50 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:52:50 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg4-2IuAvY8X27x8fuRHC5yFoZYmF2F5nMvBeQijMVRA7BNZAqo8zfRB4YXRg8Rtn9woRGnsVENU1mbuvazrfbB7Z1ZAJ_rKRBtjyh5du3wB1IpKPxEs08IK0vyR4rdpr1qbvSqL1t2-b2jHSMeAU-Dl0i2maVbglwr7QeaAhEWjZFo Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Project-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Project-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Tenant: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Tenant-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-User: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:50 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:52:50 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:52:50 user devstack@n-api.service[72506]: X-User-Id: 6c503a0141cf4794bf4e6faba59bdba3 Apr 12 01:52:50 user devstack@n-api.service[72506]: X-User-Name: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:50 user devstack@n-api.service[72506]: method: > Apr 12 01:52:50 user devstack@n-api.service[72506]: exception: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72506]: action_args: {'id': '371f55da-306a-4ef9-b04f-7c22e8071e5c'} Apr 12 01:52:50 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Returning 404 to user: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:52:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6ce4166c-9270-4746-8361-cabde3af07ae tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 404 len: 111 microversion: 2.1 time: 0.029214 Apr 12 01:52:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 73/214] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:52:50 2023] DELETE /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] [instance: 371f55da-306a-4ef9-b04f-7c22e8071e5c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] HTTP exception thrown: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Request method failure captured: Apr 12 01:52:50 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c HTTP/1.1 Apr 12 01:52:50 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:52:50 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:52:50 user devstack@n-api.service[72507]: Connection: close Apr 12 01:52:50 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:52:50 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:52:50 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:52:50 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:52:50 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg4-2IuAvY8X27x8fuRHC5yFoZYmF2F5nMvBeQijMVRA7BNZAqo8zfRB4YXRg8Rtn9woRGnsVENU1mbuvazrfbB7Z1ZAJ_rKRBtjyh5du3wB1IpKPxEs08IK0vyR4rdpr1qbvSqL1t2-b2jHSMeAU-Dl0i2maVbglwr7QeaAhEWjZFo Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Project-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Project-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2a7713a76bb546d2a348927f18b12eb7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Tenant: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Tenant-Id: 2a7713a76bb546d2a348927f18b12eb7 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-DeleteServersTestJSON-986135774 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-User: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:50 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:52:50 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:52:50 user devstack@n-api.service[72507]: X-User-Id: 6c503a0141cf4794bf4e6faba59bdba3 Apr 12 01:52:50 user devstack@n-api.service[72507]: X-User-Name: tempest-DeleteServersTestJSON-986135774-project-member Apr 12 01:52:50 user devstack@n-api.service[72507]: method: > Apr 12 01:52:50 user devstack@n-api.service[72507]: exception: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72507]: action_args: {'id': '371f55da-306a-4ef9-b04f-7c22e8071e5c'} Apr 12 01:52:50 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. Apr 12 01:52:50 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:52:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] Returning 404 to user: Instance 371f55da-306a-4ef9-b04f-7c22e8071e5c could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:52:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-15ce1afd-1c17-4149-b929-5a3a9fd99c35 tempest-DeleteServersTestJSON-986135774 tempest-DeleteServersTestJSON-986135774-project-member] 10.0.0.210 "GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c" status: 404 len: 111 microversion: 2.1 time: 0.036528 Apr 12 01:52:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 72/215] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:52:50 2023] GET /compute/v2.1/servers/371f55da-306a-4ef9-b04f-7c22e8071e5c => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-unplugged", "status": "completed", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-plugged", "status": "completed", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Creating event network-vif-unplugged:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] Creating event network-vif-plugged:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-d604541c-dc9b-4d0f-9045-8080175213bb req-12f35bdd-faab-45d5-9bd7-f40f4c6fe270 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.038994 Apr 12 01:52:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 71/216] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:50 2023] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:51 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-deleted", "tag": "de3cdf71-2461-4e2f-b11f-dba0cca9c85f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] Creating event network-vif-deleted:de3cdf71-2461-4e2f-b11f-dba0cca9c85f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-cc717d5c-bcb4-42b1-8565-e69a3e9fb3af req-563525fe-3a5a-41a8-a92a-3b983d20d014 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040304 Apr 12 01:52:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 74/217] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:51 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-deleted", "tag": "9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] Creating event network-vif-deleted:9ef8b50f-dabb-4937-a6a2-a31cae6f5bc9 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8c2e4c1d-87f8-4a7f-8950-8ba3bb264fef req-870a030e-098c-47a6-a041-95f8c5e5c14b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037637 Apr 12 01:52:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 73/218] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-deleted", "tag": "fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-deleted", "tag": "3b39d5eb-3055-44f9-8be3-4cabeddd3e3f"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-deleted", "tag": "b744b6a0-6c5b-4c1f-b73d-c893fe058df4"}, {"server_uuid": "221e01f2-18c8-468b-993b-480cdda60dd7", "name": "network-vif-deleted", "tag": "8abd0ba3-643e-45d8-ab51-c693c4b2f1f1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Creating event network-vif-deleted:fb465ea4-ce8b-43f6-bc13-0cf66dbc0d43 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Creating event network-vif-deleted:3b39d5eb-3055-44f9-8be3-4cabeddd3e3f for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Creating event network-vif-deleted:b744b6a0-6c5b-4c1f-b73d-c893fe058df4 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] Creating event network-vif-deleted:8abd0ba3-643e-45d8-ab51-c693c4b2f1f1 for instance 221e01f2-18c8-468b-993b-480cdda60dd7 on user Apr 12 01:52:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-08a4d30f-066a-4463-8858-a1cd9b1808c4 req-ba0e254a-6fa6-4ecc-b1eb-7029a0d33e3e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 696 microversion: 2.1 time: 0.037803 Apr 12 01:52:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 72/219] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:54 2023] POST /compute/v2.1/os-server-external-events => generated 696 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:55 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-deleted", "tag": "890247d7-ec5c-429e-beef-5666b669f4f3"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-deleted", "tag": "3862628b-e679-40e9-ba14-5aaab53bb289"}, {"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-deleted", "tag": "6ba40fff-9cc4-47e6-aae2-857a7b1b3445"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Creating event network-vif-deleted:890247d7-ec5c-429e-beef-5666b669f4f3 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Creating event network-vif-deleted:3862628b-e679-40e9-ba14-5aaab53bb289 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] Creating event network-vif-deleted:6ba40fff-9cc4-47e6-aae2-857a7b1b3445 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-14557e55-c9be-44c3-b6be-9c0e604568e6 req-f4d1bd6f-0734-4979-92e9-714106d08a03 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.059538 Apr 12 01:52:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 75/220] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:55 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:52:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59c7c418-902e-46b7-9302-5c630df18ca2", "name": "network-vif-deleted", "tag": "1834b43d-230e-4d08-8980-3415727f8b44"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:52:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:52:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:52:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:52:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] Creating event network-vif-deleted:1834b43d-230e-4d08-8980-3415727f8b44 for instance 59c7c418-902e-46b7-9302-5c630df18ca2 on user Apr 12 01:52:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-bd8a2cba-0eb8-4f0a-9dfb-deed883a7c9d req-7fa3aa14-da9d-442c-b704-a90c523e42d6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029953 Apr 12 01:52:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 74/221] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:52:58 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=47,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] HTTP exception thrown: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Request method failure captured: Apr 12 01:53:07 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 HTTP/1.1 Apr 12 01:53:07 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:07 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:07 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:07 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:07 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:07 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:07 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:07 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg40PD-NL9R8JqE8hvQeklsvqpMJpcWG3557cshZL5a59Fg7hdWrWMDzVQi3_oDsDT2mBnvI8IHHNb4RpzVX1Sj1lRKYTTAtwYCi7VlKQ1tAlgKXTwUfIU3MXYPwnzF04jUVDhg4MQI7qsenfPxV7gENUssRn-XgxcUIcNFtDlwEIHY Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Project-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Tenant-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:07 user devstack@n-api.service[72505]: X-User: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:07 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:07 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:07 user devstack@n-api.service[72505]: X-User-Id: e400a3359c844c449445cfba08ef7a7b Apr 12 01:53:07 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:07 user devstack@n-api.service[72505]: method: > Apr 12 01:53:07 user devstack@n-api.service[72505]: exception: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:07 user devstack@n-api.service[72505]: action_args: {'id': 'f2eedf78-0fd6-4cc1-8efd-50958e07fde6'} Apr 12 01:53:07 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Returning 404 to user: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d0641e38-49e0-4fbb-8f64-6a2fd23ccc46 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 404 len: 111 microversion: 2.1 time: 0.076416 Apr 12 01:53:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 73/222] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:06 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:53:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-92ff78dd-78b7-4034-86f0-341a6de017d3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '.version_select at 0x7fb203439a20>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-92ff78dd-78b7-4034-86f0-341a6de017d3 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-38328491" status: 202 len: 0 microversion: 2.1 time: 0.030573 Apr 12 01:53:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 76/223] 10.0.0.210 () {62 vars in 1292 bytes} [Wed Apr 12 01:53:07 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-38328491 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=22 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=39,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] HTTP exception thrown: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5160afaf-aa76-46ae-8eb9-f396d4cd9180 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.058314 Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Request method failure captured: Apr 12 01:53:10 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 HTTP/1.1 Apr 12 01:53:10 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:10 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:10 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:10 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:10 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:10 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:10 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:10 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg40PD-NL9R8JqE8hvQeklsvqpMJpcWG3557cshZL5a59Fg7hdWrWMDzVQi3_oDsDT2mBnvI8IHHNb4RpzVX1Sj1lRKYTTAtwYCi7VlKQ1tAlgKXTwUfIU3MXYPwnzF04jUVDhg4MQI7qsenfPxV7gENUssRn-XgxcUIcNFtDlwEIHY Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Project-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Tenant-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72505]: X-User: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:10 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:10 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:10 user devstack@n-api.service[72505]: X-User-Id: e400a3359c844c449445cfba08ef7a7b Apr 12 01:53:10 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:10 user devstack@n-api.service[72505]: method: > Apr 12 01:53:10 user devstack@n-api.service[72505]: exception: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72505]: action_args: {'id': 'f2eedf78-0fd6-4cc1-8efd-50958e07fde6'} Apr 12 01:53:10 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Returning 404 to user: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-19d6f798-2563-4e93-a63a-96b226fb1441 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 404 len: 111 microversion: 2.1 time: 0.030919 Apr 12 01:53:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 74/224] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:53:10 2023] DELETE /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:53:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 75/225] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:53:10 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] [instance: f2eedf78-0fd6-4cc1-8efd-50958e07fde6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] HTTP exception thrown: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Request method failure captured: Apr 12 01:53:10 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 HTTP/1.1 Apr 12 01:53:10 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:53:10 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:53:10 user devstack@n-api.service[72506]: Connection: close Apr 12 01:53:10 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:53:10 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:53:10 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:53:10 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:53:10 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg40PD-NL9R8JqE8hvQeklsvqpMJpcWG3557cshZL5a59Fg7hdWrWMDzVQi3_oDsDT2mBnvI8IHHNb4RpzVX1Sj1lRKYTTAtwYCi7VlKQ1tAlgKXTwUfIU3MXYPwnzF04jUVDhg4MQI7qsenfPxV7gENUssRn-XgxcUIcNFtDlwEIHY Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Project-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/742d0477bb3b464e8d98d42a85c330e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Tenant-Id: 742d0477bb3b464e8d98d42a85c330e7 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-1163480770 Apr 12 01:53:10 user devstack@n-api.service[72506]: X-User: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:10 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:53:10 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:53:10 user devstack@n-api.service[72506]: X-User-Id: e400a3359c844c449445cfba08ef7a7b Apr 12 01:53:10 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-1163480770-project-member Apr 12 01:53:10 user devstack@n-api.service[72506]: method: > Apr 12 01:53:10 user devstack@n-api.service[72506]: exception: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72506]: action_args: {'id': 'f2eedf78-0fd6-4cc1-8efd-50958e07fde6'} Apr 12 01:53:10 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. Apr 12 01:53:10 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] Returning 404 to user: Instance f2eedf78-0fd6-4cc1-8efd-50958e07fde6 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0819c6a8-3a3a-4dc4-812d-6374d4788e75 tempest-AttachSCSIVolumeTestJSON-1163480770 tempest-AttachSCSIVolumeTestJSON-1163480770-project-member] 10.0.0.210 "GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6" status: 404 len: 111 microversion: 2.1 time: 0.028708 Apr 12 01:53:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 77/226] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:10 2023] GET /compute/v2.1/servers/f2eedf78-0fd6-4cc1-8efd-50958e07fde6 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-842559334", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "387dd6cf-c04c-4f01-b526-b70c95a66b22"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] validate_networks() for [('387dd6cf-c04c-4f01-b526-b70c95a66b22', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:53:10 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Getting quotas for user aeb7fd40946e43dc860d63bb8958d633 and project d0dd135bd60941dbb68b3e677c4878e9. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:53:11 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:53:11 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-bddd3bec-874d-44cf-abf7-e565ff972343 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.643948 Apr 12 01:53:11 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 75/227] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:53:10 2023] POST /compute/v2.1/servers => generated 378 bytes in 645 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:53:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-dc7ab806-a3dc-431e-89f8-328b3c632199 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:11 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-dc7ab806-a3dc-431e-89f8-328b3c632199 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:11 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-dc7ab806-a3dc-431e-89f8-328b3c632199 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-dc7ab806-a3dc-431e-89f8-328b3c632199 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9" status: 200 len: 1211 microversion: 2.1 time: 0.071057 Apr 12 01:53:11 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 76/228] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:11 2023] GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 => generated 1211 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:13 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] Creating event network-changed:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:53:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-e2161f6f-8a02-48e9-9401-4f970ab1cec7 req-4da29092-fdb4-4347-a335-3d4f2b302e48 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032797 Apr 12 01:53:13 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 78/229] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:13 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:53:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-79498ef5-30ce-46d2-8a7f-23c6fd13861c req-998fb3e9-21b6-4d85-922a-7118ca6df642 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047242 Apr 12 01:53:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 76/230] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:53:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b020f019-abc2-4ba6-9535-382321a3d97c req-5dba86fe-3892-402f-b8eb-35e6e5534a7f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042533 Apr 12 01:53:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 77/231] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:28 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=7,nova_api:DELETE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:28 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1114639621", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "519f5539-952c-455f-aae3-420bb4c92b09"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] validate_networks() for [('519f5539-952c-455f-aae3-420bb4c92b09', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=19,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Getting quotas for project 32e80d0c6d6f44c786b1053c0547de3b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1600919856", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "a0d34b37-6e08-4f2e-a7df-43c3b7d8080f"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Getting quotas for user a92cd627a1c04cedacc4aa3e2398b960 and project 32e80d0c6d6f44c786b1053c0547de3b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:53:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] validate_networks() for [('a0d34b37-6e08-4f2e-a7df-43c3b7d8080f', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:53:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-227f5355-66a2-4a6a-b20b-d718d6e765a8 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.631173 Apr 12 01:53:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 79/232] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:53:28 2023] POST /compute/v2.1/servers => generated 378 bytes in 633 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-40d6c808-7eda-4e4d-8601-a5bc4d729675 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-40d6c808-7eda-4e4d-8601-a5bc4d729675 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:29 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-40d6c808-7eda-4e4d-8601-a5bc4d729675 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a92cd627a1c04cedacc4aa3e2398b960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32e80d0c6d6f44c786b1053c0547de3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-40d6c808-7eda-4e4d-8601-a5bc4d729675 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9" status: 200 len: 1208 microversion: 2.1 time: 0.079141 Apr 12 01:53:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 78/233] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:29 2023] GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 => generated 1208 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Getting quotas for project 3ce361b6fa374bca8dc0a48a53ad54f9. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Getting quotas for user 4d876b30b1724669a8d7e8bd543fa18c and project 3ce361b6fa374bca8dc0a48a53ad54f9. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:53:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:53:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b3b83a6e-f3ab-4df6-8f73-c5ce1feea565 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.546885 Apr 12 01:53:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 77/234] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:53:29 2023] POST /compute/v2.1/servers => generated 378 bytes in 548 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:53:30 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a0d9a42a-76b9-4822-b998-5de7506a82d0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:30 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a0d9a42a-76b9-4822-b998-5de7506a82d0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:30 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-a0d9a42a-76b9-4822-b998-5de7506a82d0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a0d9a42a-76b9-4822-b998-5de7506a82d0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09" status: 200 len: 1208 microversion: 2.1 time: 0.146404 Apr 12 01:53:30 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 80/235] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:30 2023] GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 => generated 1208 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:31 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0e575920-5b71-4c1c-b600-36e5bd7de707 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1503 microversion: 2.1 time: 0.169622 Apr 12 01:53:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 79/236] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:31 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1503 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] Creating event network-changed:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:53:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f73998a4-da28-4f23-9198-3727efbed563 req-ed96152f-0b36-4a40-a735-ce2984d11585 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033536 Apr 12 01:53:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 78/237] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:32 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:53:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-dac03d22-b162-4dc7-a806-e52d32874296 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 204 len: 0 microversion: 2.1 time: 0.141025 Apr 12 01:53:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 81/238] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:53:32 2023] DELETE /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] Creating event network-changed:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:53:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-ce83c779-17e5-49d0-a7ff-0b329c05e3d7 req-214fc884-ce5a-4f4e-824d-801535561e71 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048170 Apr 12 01:53:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 79/239] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:32 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:33 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-af30adba-156b-4375-afb6-409c60107e8f tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 200 len: 1499 microversion: 2.1 time: 0.348646 Apr 12 01:53:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 80/240] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:32 2023] GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 1499 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "name": "network-vif-unplugged", "status": "completed", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}, {"server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "name": "network-vif-plugged", "status": "completed", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Creating event network-vif-unplugged:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:53:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] Creating event network-vif-plugged:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:53:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-61275789-2430-48bb-bd39-086a4dd7545f req-c6797ffb-689a-4da1-ba64-661fcfdaba30 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.033281 Apr 12 01:53:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 82/241] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:34 2023] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:34 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b865f0d3-695b-40ee-82df-d1b3f7edc047 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1657 microversion: 2.1 time: 0.182307 Apr 12 01:53:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 80/242] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:34 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1657 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "333bfdb2-c5a0-443a-bcc1-c203e669c896"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-24d1b12e-54b8-48df-9e91-762770098c10 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.266638 Apr 12 01:53:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 81/243] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:53:34 2023] POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments => generated 194 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f32ace1-1adc-4b75-a1cc-daac2e724846", "name": "network-vif-deleted", "tag": "98ce60a7-28d6-42db-8002-f433bd77175e"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] Creating event network-vif-deleted:98ce60a7-28d6-42db-8002-f433bd77175e for instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 on user Apr 12 01:53:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-95bb446d-f996-4154-97ee-b3d23d60e4e6 req-bfd79cf5-ed01-4ce5-a65e-73770e9094c4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039711 Apr 12 01:53:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 83/244] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:35 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-81d15de1-6d79-4263-a0b9-93f2db646f92 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.070076 Apr 12 01:53:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 81/245] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:53:35 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02/os-volume_attachments => generated 25 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:53:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "name": "network-vif-plugged", "status": "completed", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] Creating event network-vif-plugged:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:53:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-90fbdc0c-10db-483e-a13d-6967ee62940b req-b10ae652-824a-419e-b8db-df14fcfe655d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057284 Apr 12 01:53:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 82/246] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:35 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] HTTP exception thrown: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Request method failure captured: Apr 12 01:53:36 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 HTTP/1.1 Apr 12 01:53:36 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:36 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:36 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:36 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:36 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:36 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:36 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:36 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg5Cp0rIjC-AmEikM7fGU6ZgavGLui-1YGVffdHQnvJ4mnH-UevkXiXZzfhEcvJCpDP-Z4wN6GiP8WMrSeYMhob25bZTX9CFJFnLQBTy5FxkPp8UEaXlfBltkoaq3eoilzx8zbno038VJTe889q-xiKgCqYY_LWE6rJXHfFpKsnKn6U Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.32 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Project-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Tenant-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-User: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:36 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:36 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:36 user devstack@n-api.service[72505]: X-User-Id: f288eba0f612445596edccf43f08fa22 Apr 12 01:53:36 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:36 user devstack@n-api.service[72505]: method: > Apr 12 01:53:36 user devstack@n-api.service[72505]: exception: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:36 user devstack@n-api.service[72505]: action_args: {'id': '221e01f2-18c8-468b-993b-480cdda60dd7'} Apr 12 01:53:36 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:36 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Returning 404 to user: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:36 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5d8c28de-3242-432f-949b-06a29c91f3f3 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 404 len: 111 microversion: 2.32 time: 0.033270 Apr 12 01:53:36 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 82/247] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:53:36 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-740da9f8-6612-4dff-bb8a-b902ef2019f9 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02" status: 204 len: 0 microversion: 2.1 time: 0.153338 Apr 12 01:53:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 84/248] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:53:35 2023] DELETE /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:53:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9edae300-5a5d-4c30-b7b9-6c1ac4cc697e tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '.version_select at 0x7fb20349e4d0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-9edae300-5a5d-4c30-b7b9-6c1ac4cc697e tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1357527151" status: 204 len: 0 microversion: 2.32 time: 0.025315 Apr 12 01:53:36 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 83/249] 10.0.0.210 () {64 vars in 1339 bytes} [Wed Apr 12 01:53:36 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1357527151 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:53:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "name": "network-vif-plugged", "status": "completed", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] Creating event network-vif-plugged:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:53:36 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '592f78cbcf7046e68bcf6c88e039a9be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3a1f516270354a82995f75c4c149dad2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-40cd04a1-e72e-4710-a98b-12b5ed7da919 req-878ce747-38aa-4f37-aa20-ca98a7d97e4a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045100 Apr 12 01:53:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 85/250] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:36 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:36 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-175bdc03-7b69-47c5-8b73-aa5810dcbe52 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02" status: 200 len: 1648 microversion: 2.1 time: 0.311640 Apr 12 01:53:36 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 83/251] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:36 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 => generated 1648 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:53:37 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "name": "network-vif-deleted", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] Creating event network-vif-deleted:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:53:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-46a1f2e3-d5f2-41fa-a00a-ae1974e49dab req-d35856d2-6d6b-49cd-84c9-ba8269c65a5b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049886 Apr 12 01:53:37 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 84/252] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:37 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "name": "network-vif-plugged", "status": "completed", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] Creating event network-vif-plugged:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-94e3cc2f-7b98-44db-9302-f4c8dcb643fa req-a44aced3-3fab-40b2-b1be-1bf9ac55fd77 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051788 Apr 12 01:53:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 86/253] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:37 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] HTTP exception thrown: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Request method failure captured: Apr 12 01:53:38 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 HTTP/1.1 Apr 12 01:53:38 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:38 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:38 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:38 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:38 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:38 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:38 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:38 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg5CH6MRGv-NkDdq95Qp_Vd0UOTs391Rx8JFxt5liIcYVkuu7KyEzk7Ikoqu3b-BFWgqhMWh24szcGbrfRX37eHY1btN1TJslQQs6Wc6n7A1hUHBV7uKaFDN2168_n4Tg6snG5SbBDNjrIUNkdU0bMg8ewY1xlqehWFXbrXg8_In2M8 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.42 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Project-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/85c85b2820514000b08e47046f7bd717"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Tenant-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:38 user devstack@n-api.service[72505]: X-User: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:38 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:38 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:38 user devstack@n-api.service[72505]: X-User-Id: aa19aa56b6ee4a10a07eb6e421a131bd Apr 12 01:53:38 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:38 user devstack@n-api.service[72505]: method: > Apr 12 01:53:38 user devstack@n-api.service[72505]: exception: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:38 user devstack@n-api.service[72505]: action_args: {'id': '59c7c418-902e-46b7-9302-5c630df18ca2'} Apr 12 01:53:38 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:38 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Returning 404 to user: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a38831d5-2087-4fe7-98da-267c30576e89 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 404 len: 111 microversion: 2.42 time: 0.056610 Apr 12 01:53:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 84/254] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:53:38 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-225a0863-56ca-4653-b235-b959621e060a tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '.version_select at 0x7fb20349d480>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-225a0863-56ca-4653-b235-b959621e060a tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1066267509" status: 204 len: 0 microversion: 2.42 time: 0.023052 Apr 12 01:53:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 85/255] 10.0.0.210 () {64 vars in 1339 bytes} [Wed Apr 12 01:53:38 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1066267509 => generated 0 bytes in 24 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "name": "network-vif-plugged", "status": "completed", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}, {"server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "name": "network-vif-unplugged", "status": "completed", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}, {"server_uuid": "34cbdbd9-16ac-4cbc-be28-c88b7c23de02", "name": "network-vif-plugged", "status": "completed", "tag": "0afbc2d8-20bd-4f40-b45b-53bad28276d4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Creating event network-vif-plugged:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Creating event network-vif-unplugged:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] Creating event network-vif-plugged:0afbc2d8-20bd-4f40-b45b-53bad28276d4 for instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 on user Apr 12 01:53:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4b96035c-10a2-40fc-993f-c26e0ec71094 req-61ad34e4-0bdf-4651-bcec-6865f842da5f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 527 microversion: 2.1 time: 0.060214 Apr 12 01:53:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 87/256] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:38 2023] POST /compute/v2.1/os-server-external-events => generated 527 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] HTTP exception thrown: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Request method failure captured: Apr 12 01:53:40 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 HTTP/1.1 Apr 12 01:53:40 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:40 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:40 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:40 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:40 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:40 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:40 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:40 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg5Cp0rIjC-AmEikM7fGU6ZgavGLui-1YGVffdHQnvJ4mnH-UevkXiXZzfhEcvJCpDP-Z4wN6GiP8WMrSeYMhob25bZTX9CFJFnLQBTy5FxkPp8UEaXlfBltkoaq3eoilzx8zbno038VJTe889q-xiKgCqYY_LWE6rJXHfFpKsnKn6U Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.32 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Project-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Tenant-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-User: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:40 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:40 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:40 user devstack@n-api.service[72505]: X-User-Id: f288eba0f612445596edccf43f08fa22 Apr 12 01:53:40 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:40 user devstack@n-api.service[72505]: method: > Apr 12 01:53:40 user devstack@n-api.service[72505]: exception: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72505]: action_args: {'id': '221e01f2-18c8-468b-993b-480cdda60dd7'} Apr 12 01:53:40 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Returning 404 to user: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-af6872db-44e8-488c-b739-3468f534aaf4 tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 404 len: 111 microversion: 2.32 time: 0.051289 Apr 12 01:53:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 85/257] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:53:40 2023] DELETE /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] [instance: 221e01f2-18c8-468b-993b-480cdda60dd7] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] HTTP exception thrown: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Request method failure captured: Apr 12 01:53:40 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 HTTP/1.1 Apr 12 01:53:40 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:53:40 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:53:40 user devstack@n-api.service[72507]: Connection: close Apr 12 01:53:40 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:53:40 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:53:40 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:53:40 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:53:40 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg5Cp0rIjC-AmEikM7fGU6ZgavGLui-1YGVffdHQnvJ4mnH-UevkXiXZzfhEcvJCpDP-Z4wN6GiP8WMrSeYMhob25bZTX9CFJFnLQBTy5FxkPp8UEaXlfBltkoaq3eoilzx8zbno038VJTe889q-xiKgCqYY_LWE6rJXHfFpKsnKn6U Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.32 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Project-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Project-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3071310920ea42f9a66e1a92d6ecd324"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Tenant: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Tenant-Id: 3071310920ea42f9a66e1a92d6ecd324 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1403588062 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-User: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:40 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:53:40 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:53:40 user devstack@n-api.service[72507]: X-User-Id: f288eba0f612445596edccf43f08fa22 Apr 12 01:53:40 user devstack@n-api.service[72507]: X-User-Name: tempest-TaggedBootDevicesTest-1403588062-project-member Apr 12 01:53:40 user devstack@n-api.service[72507]: method: > Apr 12 01:53:40 user devstack@n-api.service[72507]: exception: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72507]: action_args: {'id': '221e01f2-18c8-468b-993b-480cdda60dd7'} Apr 12 01:53:40 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. Apr 12 01:53:40 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:53:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] Returning 404 to user: Instance 221e01f2-18c8-468b-993b-480cdda60dd7 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4d76c9b5-18ff-4844-95eb-e599f45ea97a tempest-TaggedBootDevicesTest-1403588062 tempest-TaggedBootDevicesTest-1403588062-project-member] 10.0.0.210 "GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7" status: 404 len: 111 microversion: 2.32 time: 0.046712 Apr 12 01:53:40 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 86/258] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:53:40 2023] GET /compute/v2.1/servers/221e01f2-18c8-468b-993b-480cdda60dd7 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] HTTP exception thrown: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Request method failure captured: Apr 12 01:53:41 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 HTTP/1.1 Apr 12 01:53:41 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:53:41 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:53:41 user devstack@n-api.service[72506]: Connection: close Apr 12 01:53:41 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:53:41 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:53:41 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:53:41 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:53:41 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg5CH6MRGv-NkDdq95Qp_Vd0UOTs391Rx8JFxt5liIcYVkuu7KyEzk7Ikoqu3b-BFWgqhMWh24szcGbrfRX37eHY1btN1TJslQQs6Wc6n7A1hUHBV7uKaFDN2168_n4Tg6snG5SbBDNjrIUNkdU0bMg8ewY1xlqehWFXbrXg8_In2M8 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Openstack-Nova-Api-Version: 2.42 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Project-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/85c85b2820514000b08e47046f7bd717"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Tenant-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72506]: X-User: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:41 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:53:41 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:53:41 user devstack@n-api.service[72506]: X-User-Id: aa19aa56b6ee4a10a07eb6e421a131bd Apr 12 01:53:41 user devstack@n-api.service[72506]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:41 user devstack@n-api.service[72506]: method: > Apr 12 01:53:41 user devstack@n-api.service[72506]: exception: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72506]: action_args: {'id': '59c7c418-902e-46b7-9302-5c630df18ca2'} Apr 12 01:53:41 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Returning 404 to user: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-25eb6749-7d5a-48ea-b184-86dad96761b0 tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 404 len: 111 microversion: 2.42 time: 0.041621 Apr 12 01:53:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 88/259] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:53:41 2023] DELETE /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] [instance: 59c7c418-902e-46b7-9302-5c630df18ca2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] HTTP exception thrown: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Request method failure captured: Apr 12 01:53:41 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 HTTP/1.1 Apr 12 01:53:41 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:53:41 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:53:41 user devstack@n-api.service[72505]: Connection: close Apr 12 01:53:41 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:53:41 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:53:41 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:53:41 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:53:41 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg5CH6MRGv-NkDdq95Qp_Vd0UOTs391Rx8JFxt5liIcYVkuu7KyEzk7Ikoqu3b-BFWgqhMWh24szcGbrfRX37eHY1btN1TJslQQs6Wc6n7A1hUHBV7uKaFDN2168_n4Tg6snG5SbBDNjrIUNkdU0bMg8ewY1xlqehWFXbrXg8_In2M8 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.42 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Project-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/85c85b2820514000b08e47046f7bd717"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/85c85b2820514000b08e47046f7bd717"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Tenant-Id: 85c85b2820514000b08e47046f7bd717 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1956984568 Apr 12 01:53:41 user devstack@n-api.service[72505]: X-User: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:41 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:53:41 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:53:41 user devstack@n-api.service[72505]: X-User-Id: aa19aa56b6ee4a10a07eb6e421a131bd Apr 12 01:53:41 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1956984568-project-member Apr 12 01:53:41 user devstack@n-api.service[72505]: method: > Apr 12 01:53:41 user devstack@n-api.service[72505]: exception: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72505]: action_args: {'id': '59c7c418-902e-46b7-9302-5c630df18ca2'} Apr 12 01:53:41 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. Apr 12 01:53:41 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:53:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] Returning 404 to user: Instance 59c7c418-902e-46b7-9302-5c630df18ca2 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:53:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e15cec58-6d98-44cb-ae74-32e29595f34d tempest-TaggedBootDevicesTest_v242-1956984568 tempest-TaggedBootDevicesTest_v242-1956984568-project-member] 10.0.0.210 "GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2" status: 404 len: 111 microversion: 2.42 time: 0.065475 Apr 12 01:53:41 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 86/260] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:53:41 2023] GET /compute/v2.1/servers/59c7c418-902e-46b7-9302-5c630df18ca2 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:42 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-408ca5d4-8baa-4c83-92a6-d946f29a3882 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1703 microversion: 2.1 time: 0.210217 Apr 12 01:53:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 87/261] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:42 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1703 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"rebuild": {"name": "tempest-ServerActionsTestJSON-server-1476053452", "metadata": {"rebuild": "server"}, "adminPass": "***", "imageRef": "}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: dict_keys(['metadata_items']) {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: dict_keys(['metadata_items']) {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-71782933-b15b-46cc-a51a-a6bc72a51bed tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action" status: 202 len: 1159 microversion: 2.1 time: 0.468165 Apr 12 01:53:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 89/262] 10.0.0.210 () {64 vars in 1356 bytes} [Wed Apr 12 01:53:42 2023] POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action => generated 1159 bytes in 470 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:43 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-54a25110-d2c7-4cb9-a1f7-7fb7a8ae8e93 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1732 microversion: 2.1 time: 0.290899 Apr 12 01:53:43 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 87/263] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:43 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1732 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:53:44 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-unplugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] Creating event network-vif-unplugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:53:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e12a91aa-07ff-4604-84fd-1d56e7162a15 req-3d6330f0-8673-49b8-a0d0-3a439a8537e2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044335 Apr 12 01:53:44 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 88/264] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:44 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:53:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-e6808bf9-74f5-491b-94c3-7b975c12a615 req-15cec2e0-a05c-4ddf-8125-9d6ae3532c90 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035143 Apr 12 01:53:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 90/265] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:46 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:49 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Going to try to start instance {{(pid=72505) start /opt/stack/nova/nova/compute/api.py:2787}} Apr 12 01:53:49 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-36b37606-973a-4ae3-83af-a21421ab0988 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action" status: 202 len: 0 microversion: 2.1 time: 0.145143 Apr 12 01:53:49 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 88/266] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:53:48 2023] POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:49 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:53:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-897ae543-e939-4dd2-83a9-0170a34ece6e tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1697 microversion: 2.1 time: 0.320561 Apr 12 01:53:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 89/267] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:53:49 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1697 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:53:54 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:54 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:54 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:54 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:54 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:53:54 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-607c2fdf-344f-41d9-9530-a9a1255b56fb req-99f7d85a-a908-4fd3-8998-948c98239249 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039145 Apr 12 01:53:54 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 91/268] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:54 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:53:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}, {"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}, {"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:53:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:53:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:53:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:53:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:53:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:53:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:53:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-53d76082-7907-402e-95a5-0148f7d87953 req-b3840f7d-7af5-49cf-a2c8-e7c97dca5d03 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.039493 Apr 12 01:53:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 89/269] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:53:56 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=28,nova_api:DELETE=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:01 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f827df8b-cd6f-4fd9-8206-05d1be258852 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9" status: 200 len: 1495 microversion: 2.1 time: 0.326791 Apr 12 01:54:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 90/270] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:01 2023] GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 => generated 1495 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:03 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e8607d02-5950-403d-9a83-c67959a411b5 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'create', calling method: .version_select at 0x7fb2032d2a70>, body: {"keypair": {"name": "tempest-keypair-682814124", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGI/RFUfuZXI6dJ1hjD1JW2D0k8xZh+epQO+5dsrL15C9zRJbRnL07EyJ7CDIspWuf9JFtZqO5PUPIpDarEwCZLJxygT+Ba5rtS558RsdJ4Kzf5y9GMjIllAHF5H0rE0pA=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:03 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:03 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-e8607d02-5950-403d-9a83-c67959a411b5 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Getting quotas for project cdf7e36a5e4b44f690f875999b4b101a. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:54:03 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-e8607d02-5950-403d-9a83-c67959a411b5 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Getting quotas for user e6c9d8100d504d1c8437b627f5564e33 and project cdf7e36a5e4b44f690f875999b4b101a. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:54:03 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e8607d02-5950-403d-9a83-c67959a411b5 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.161620 Apr 12 01:54:03 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 92/271] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:54:03 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerStableDeviceRescueTest-server-1700354406", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "849f67cf-2fe1-4792-ac08-a770ab4435c3"}], "security_groups": [{"name": "tempest-securitygroup--886833994"}], "key_name": "tempest-keypair-682814124", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] validate_networks() for [('849f67cf-2fe1-4792-ac08-a770ab4435c3', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Getting quotas for project cdf7e36a5e4b44f690f875999b4b101a. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Getting quotas for user e6c9d8100d504d1c8437b627f5564e33 and project cdf7e36a5e4b44f690f875999b4b101a. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:54:05 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:54:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2255f9dd-eaa5-46ae-af4e-7ba39e4294f7 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 0.567349 Apr 12 01:54:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 90/272] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:54:05 2023] POST /compute/v2.1/servers => generated 403 bytes in 569 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:54:05 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-77623768-ffb2-4429-9948-b303e506a03d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:05 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-77623768-ffb2-4429-9948-b303e506a03d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:05 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-77623768-ffb2-4429-9948-b303e506a03d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:05 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-77623768-ffb2-4429-9948-b303e506a03d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1239 microversion: 2.1 time: 0.090014 Apr 12 01:54:05 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 91/273] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:05 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1239 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=8,nova_cell1:INSERT=4,nova_cell1:DELETE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] Creating event network-changed:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:54:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-b4cd1a96-29d2-4112-b24f-f46d41b8b158 req-69c285e8-20a6-4caf-828b-f2d37d4a75d4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036672 Apr 12 01:54:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 93/274] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:08 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:54:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-edd44df3-da31-4bd2-afac-60d6329bf039 req-29f8cccb-7800-422a-8d7a-5c567d70710a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045744 Apr 12 01:54:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 91/275] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:10 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:12 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:54:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-0b88db77-35c7-469d-b5d5-9ec15cf0d5a7 req-6bc25868-a8cf-4119-b42a-9d42b94ee3c6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034577 Apr 12 01:54:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 92/276] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:12 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=12,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:19 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a92cd627a1c04cedacc4aa3e2398b960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32e80d0c6d6f44c786b1053c0547de3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e16f035d-f849-4006-982d-a93bd874260d tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9" status: 200 len: 1486 microversion: 2.1 time: 0.161673 Apr 12 01:54:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 94/277] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:19 2023] GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 => generated 1486 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:20 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8ede141b-875f-4f30-aade-ea06eb75c2c4 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09" status: 200 len: 1485 microversion: 2.1 time: 0.190105 Apr 12 01:54:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 92/278] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:20 2023] GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 => generated 1485 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] HTTP exception thrown: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:54:23 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 HTTP/1.1 Apr 12 01:54:23 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:54:23 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:54:23 user devstack@n-api.service[72507]: Connection: close Apr 12 01:54:23 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:54:23 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:54:23 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:54:23 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:54:23 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:54:23 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:54:23 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:54:23 user devstack@n-api.service[72507]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:54:23 user devstack@n-api.service[72507]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:54:23 user devstack@n-api.service[72507]: method: > Apr 12 01:54:23 user devstack@n-api.service[72507]: exception: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:54:23 user devstack@n-api.service[72507]: action_args: {'id': '2f32ace1-1adc-4b75-a1cc-daac2e724846'} Apr 12 01:54:23 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:54:23 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:54:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 404 to user: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:54:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f9f69a7d-cc62-442d-87da-988e950fcb9b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 404 len: 111 microversion: 2.1 time: 0.042931 Apr 12 01:54:23 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 93/279] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:23 2023] GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:23 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-367234e3-ff34-46c1-8567-72868597f390 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1491 microversion: 2.1 time: 0.170952 Apr 12 01:54:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 95/280] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:23 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1491 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] [instance: 34cbdbd9-16ac-4cbc-be28-c88b7c23de02] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] HTTP exception thrown: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Request method failure captured: Apr 12 01:54:26 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 HTTP/1.1 Apr 12 01:54:26 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:54:26 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:54:26 user devstack@n-api.service[72505]: Connection: close Apr 12 01:54:26 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:54:26 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:54:26 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:54:26 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:54:26 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg45NhEWWhgwX4q8IZj1hfjXaIkPMmZMXX5Y7Ox1TAa88KmrW1F5yV2PLVdvA9Zsie9rbxDtlAGvfWNGrT2mrrT3PF6yKG6zAinUuInNPAVIe7FZW5oiG75YtSIZlrdDLxqfEKNWIzm35ChXvo0DoprMRCzZHXaaySf7CwoYxGB1L_s Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Project-Id: 3a1f516270354a82995f75c4c149dad2 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeSwap-706858033 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3a1f516270354a82995f75c4c149dad2"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3a1f516270354a82995f75c4c149dad2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3a1f516270354a82995f75c4c149dad2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeSwap-706858033 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Tenant-Id: 3a1f516270354a82995f75c4c149dad2 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeSwap-706858033 Apr 12 01:54:26 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeSwap-706858033-project-member Apr 12 01:54:26 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:54:26 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:54:26 user devstack@n-api.service[72505]: X-User-Id: 592f78cbcf7046e68bcf6c88e039a9be Apr 12 01:54:26 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeSwap-706858033-project-member Apr 12 01:54:26 user devstack@n-api.service[72505]: method: > Apr 12 01:54:26 user devstack@n-api.service[72505]: exception: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. Apr 12 01:54:26 user devstack@n-api.service[72505]: action_args: {'id': '34cbdbd9-16ac-4cbc-be28-c88b7c23de02'} Apr 12 01:54:26 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. Apr 12 01:54:26 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:54:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Returning 404 to user: Instance 34cbdbd9-16ac-4cbc-be28-c88b7c23de02 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:54:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a1782e82-a98f-4a37-ac09-3b3527bce2fe tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02" status: 404 len: 111 microversion: 2.1 time: 0.039677 Apr 12 01:54:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 93/281] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:26 2023] GET /compute/v2.1/servers/34cbdbd9-16ac-4cbc-be28-c88b7c23de02 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:54:26 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2ab99ad1-d3a1-4a09-8f43-5bbff1427108 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] Calling method '.version_select at 0x7fb20349c9d0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:26 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2ab99ad1-d3a1-4a09-8f43-5bbff1427108 tempest-TestVolumeSwap-706858033 tempest-TestVolumeSwap-706858033-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2091713473" status: 202 len: 0 microversion: 2.1 time: 0.022639 Apr 12 01:54:26 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 94/282] 10.0.0.210 () {62 vars in 1298 bytes} [Wed Apr 12 01:54:26 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2091713473 => generated 0 bytes in 24 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lazy-loading 'flavor' on Instance uuid fdbd8add-ac0e-42ce-9b34-5878c6bec208 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:54:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7bb219d7-e0f1-4699-943d-560190607343 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/action" status: 200 len: 32 microversion: 2.1 time: 0.169807 Apr 12 01:54:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 96/283] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:54:33 2023] POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/action => generated 32 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:33 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-166e46b0-c07b-4dec-80e1-afbba6e1f08b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1509 microversion: 2.1 time: 0.310540 Apr 12 01:54:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 94/284] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:33 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1509 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:34 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3233a713-2f43-4180-88b3-b16068d8ce1b tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.252485 Apr 12 01:54:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 95/285] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:33 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-unplugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] Creating event network-vif-unplugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:54:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-464d120b-7816-4377-adde-4b3d405e77f4 req-8a920eb2-9e67-410a-b27b-5832a7c50f87 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046148 Apr 12 01:54:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 97/286] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:35 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=23,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:37 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=18 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:54:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-dc0025eb-6ab0-4301-b565-fee55b0c0283 req-d992e79e-5925-4e22-872d-c73c0fe09eed service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035857 Apr 12 01:54:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 95/287] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:37 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}, {"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:54:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:54:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-1242832d-7bb1-483e-8848-a2fdd0817d58 req-859d49a6-cc49-4888-9d6f-3f4dbe6cd736 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.033110 Apr 12 01:54:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 96/288] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:54:39 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:39 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-721e34e6-c166-4fc9-94c4-a4e51f0ab484 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1701 microversion: 2.1 time: 0.325847 Apr 12 01:54:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 98/289] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:39 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1701 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:54:55 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:54:55 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-afe5ff8d-8de6-4d99-8359-6f23940efe8b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1553 microversion: 2.1 time: 0.168141 Apr 12 01:54:55 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 96/290] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:54:55 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1553 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:54:58 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=9,nova_api:DELETE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:58 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:54:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-041008c1-fe76-4131-8657-1d60782cbc52 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Action: 'create', calling method: .version_select at 0x7fb203404a60>, body: {"keypair": {"name": "tempest-keypair-1476535441", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBlxYLGiDT3ULykrJ/16FAYLe9mlS7WpoHsNkE+Oww0i5S0/0HEU1ZMf1BlJ8O7u24iaXO1qwL8Qcg5/TeN5uRr2keoVwu2xaLtJctivU4Ipg77M6FqVNyx8y2JxOGS9rQ=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:54:58 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-041008c1-fe76-4131-8657-1d60782cbc52 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Getting quotas for project 3443faad6b314920a85b8491603e30ac. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:54:58 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-041008c1-fe76-4131-8657-1d60782cbc52 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Getting quotas for user 43819af3565d4eff92ce1a51f86df4f2 and project 3443faad6b314920a85b8491603e30ac. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:54:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-041008c1-fe76-4131-8657-1d60782cbc52 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.49 time: 0.051801 Apr 12 01:54:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 97/291] 10.0.0.210 () {66 vars in 1277 bytes} [Wed Apr 12 01:54:58 2023] POST /compute/v2.1/os-keypairs => generated 401 bytes in 53 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-211209472", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "config_drive": true, "networks": [{"uuid": "988b3410-7746-4c36-99ce-5bab176cefe3"}], "security_groups": [{"name": "tempest-securitygroup--1232281572"}], "key_name": "tempest-keypair-1476535441", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=35,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] validate_networks() for [('988b3410-7746-4c36-99ce-5bab176cefe3', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Getting quotas for project 3443faad6b314920a85b8491603e30ac. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Getting quotas for user 43819af3565d4eff92ce1a51f86df4f2 and project 3443faad6b314920a85b8491603e30ac. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:55:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:55:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-aea80848-b6df-4b2f-a36d-e7487d82a4a9 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.49 time: 0.642412 Apr 12 01:55:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 99/292] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 01:55:00 2023] POST /compute/v2.1/servers => generated 404 bytes in 644 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 01:55:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-bf1766d4-3b72-4d94-ba5d-ba47e91b026d tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 200 len: 1320 microversion: 2.49 time: 0.076490 Apr 12 01:55:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 97/293] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:55:00 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 1320 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "577b8cd6-e41c-44fb-9299-7f0104fc612c"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] HTTP exception thrown: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Request method failure captured: Apr 12 01:55:03 user devstack@n-api.service[72507]: request: PUT /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9/os-volume_attachments/ffa4414e-dd3f-400a-927d-21f05c6b24ca HTTP/1.1 Apr 12 01:55:03 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:55:03 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:55:03 user devstack@n-api.service[72507]: Connection: close Apr 12 01:55:03 user devstack@n-api.service[72507]: Content-Length: 74 Apr 12 01:55:03 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:55:03 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:55:03 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:55:03 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg5UvNwlL2HfJvouJSPFlA70KbFzpbR54yZdOYTf2O4RCztQ4EXaY_isxAYv6gRvvlB5VnkDPHcBwobbudzUTyZ2wOM2sc9NfXUE52BhBoKIu53heAEM92sJrqpIDjSuDcYrWMJidDfpkZjZsznwXMHxwhSyg6Rt8mVHD9a9RSIC_5M Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Project-Id: 37f0f8a76a6a4ec6a9caa26de4805b6f Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Role: member,reader,admin Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Roles: member,reader,admin Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/37f0f8a76a6a4ec6a9caa26de4805b6f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Tenant-Id: 37f0f8a76a6a4ec6a9caa26de4805b6f Apr 12 01:55:03 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-807677884 Apr 12 01:55:03 user devstack@n-api.service[72507]: X-User: tempest-VolumesAdminNegativeTest-807677884-project-admin Apr 12 01:55:03 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:55:03 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:55:03 user devstack@n-api.service[72507]: X-User-Id: 4c7379a5c55645669ac6cd63053f58e8 Apr 12 01:55:03 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesAdminNegativeTest-807677884-project-admin Apr 12 01:55:03 user devstack@n-api.service[72507]: Apr 12 01:55:03 user devstack@n-api.service[72507]: {"volumeAttachment": {"volumeId": "577b8cd6-e41c-44fb-9299-7f0104fc612c"}} Apr 12 01:55:03 user devstack@n-api.service[72507]: method: > Apr 12 01:55:03 user devstack@n-api.service[72507]: exception: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. Apr 12 01:55:03 user devstack@n-api.service[72507]: action_args: {'server_id': '2acd2ee1-172a-4984-85ed-52a8e78aafb9', 'id': 'ffa4414e-dd3f-400a-927d-21f05c6b24ca', 'body': {'volumeAttachment': {'volumeId': '577b8cd6-e41c-44fb-9299-7f0104fc612c'}}} Apr 12 01:55:03 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. (HTTP 404) (Request-ID: req-053fd1a4-94ac-45e2-8f51-ff0809f6694b) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 430, in _update_volume_swap Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi old_volume = self.volume_api.get(context, old_volume_id) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 432, in _update_volume_swap Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. Apr 12 01:55:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] Returning 404 to user: Volume ffa4414e-dd3f-400a-927d-21f05c6b24ca could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:55:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-77822162-0169-4416-8805-37e99b640aab tempest-VolumesAdminNegativeTest-807677884 tempest-VolumesAdminNegativeTest-807677884-project-admin] 10.0.0.210 "PUT /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9/os-volume_attachments/ffa4414e-dd3f-400a-927d-21f05c6b24ca" status: 404 len: 109 microversion: 2.1 time: 0.064115 Apr 12 01:55:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 98/294] 10.0.0.210 () {64 vars in 1510 bytes} [Wed Apr 12 01:55:03 2023] PUT /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9/os-volume_attachments/ffa4414e-dd3f-400a-927d-21f05c6b24ca => generated 109 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:55:04 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] Creating event network-changed:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:55:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c51d8045-a2a1-4644-8409-f1918e05459f req-d035f468-3380-4350-804e-4bae7d494057 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094216 Apr 12 01:55:04 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 100/295] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:04 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=7 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:55:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-03f71882-59ab-4f4f-aa7a-53f3b7f14e90 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9" status: 204 len: 0 microversion: 2.1 time: 0.162892 Apr 12 01:55:06 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 98/296] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:55:06 2023] DELETE /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] Creating event network-vif-plugged:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:55:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-a42ddc7a-5118-4d18-8afd-e7341682fddd req-43aaf56a-6879-418b-8b4b-e17925e1e8de service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.266206 Apr 12 01:55:06 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 99/297] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:06 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:06 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:06 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5a29aa3f-af4e-458d-b522-fc02dd6a8b84 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9" status: 200 len: 1501 microversion: 2.1 time: 0.280985 Apr 12 01:55:06 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 101/298] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:06 2023] GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 => generated 1501 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-unplugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] Creating event network-vif-unplugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-9e24a2ba-6fd4-416a-9aa2-0ba78e3bf86d req-40640996-2a8e-490c-98aa-03dd1e1c3e86 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038207 Apr 12 01:55:06 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 99/299] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:06 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] Creating event network-vif-plugged:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:55:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-742e7a90-cda4-41c5-bc85-2f686592025d req-c6f09116-226d-467d-a8bd-ae8901cb2ae7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032054 Apr 12 01:55:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 100/300] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:08 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}, {"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}, {"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}, {"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-plugged", "status": "completed", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}, {"server_uuid": "2acd2ee1-172a-4984-85ed-52a8e78aafb9", "name": "network-vif-deleted", "tag": "d01d48de-8685-4de9-a1d2-e13833a4b274"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Creating event network-vif-plugged:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] Creating event network-vif-deleted:d01d48de-8685-4de9-a1d2-e13833a4b274 for instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 on user Apr 12 01:55:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-1ce0a743-1bf2-4ea5-9cd0-834a3006d666 req-18fca5ac-1d27-4894-a06d-2efe56c0a76f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 867 microversion: 2.1 time: 0.036183 Apr 12 01:55:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 102/301] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:08 2023] POST /compute/v2.1/os-server-external-events => generated 867 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:21 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:55:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b6ba98eb-a9cf-43a6-a95f-335172e816a1 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09" status: 204 len: 0 microversion: 2.1 time: 0.140031 Apr 12 01:55:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 100/302] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:55:21 2023] DELETE /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:21 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-31b15639-e8e8-48ac-842b-b962cd619de0 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09" status: 200 len: 1491 microversion: 2.1 time: 0.261785 Apr 12 01:55:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 101/303] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:21 2023] GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 => generated 1491 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "name": "network-vif-unplugged", "status": "completed", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] Creating event network-vif-unplugged:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:55:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ff2af2b5-2a87-4000-9b93-475a35909902 req-e5a72f7b-12bf-4001-8279-efc0ec0d4958 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046058 Apr 12 01:55:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 103/304] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:21 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:22 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "name": "network-vif-deleted", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:22 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] Creating event network-vif-deleted:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:55:22 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-77ff1df6-4012-4c6f-ad81-d9d3cf588d53 req-94becc71-c1c6-467c-b709-8af33d65da70 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030964 Apr 12 01:55:22 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 101/305] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:22 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9fa9bdbe-ee75-425e-bed5-049fdf908d09", "name": "network-vif-plugged", "status": "completed", "tag": "0a900b27-d2bd-420d-a5cf-1df8fee01d93"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] Creating event network-vif-plugged:0a900b27-d2bd-420d-a5cf-1df8fee01d93 for instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 on user Apr 12 01:55:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-76208571-070f-4397-b740-736e072cd48d req-bc2f3567-7ef6-4d5e-b1f7-0010088b3adb service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036714 Apr 12 01:55:23 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 102/306] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:23 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:24 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8b6e4c1a-023b-4715-a785-b268591462e1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1489 microversion: 2.1 time: 0.586831 Apr 12 01:55:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 104/307] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:23 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1489 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:24 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:55:24 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f33c8e95-9944-4ea7-aeb7-9e1fdaf56188 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9" status: 204 len: 0 microversion: 2.1 time: 0.142709 Apr 12 01:55:24 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 102/308] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:55:24 2023] DELETE /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a92cd627a1c04cedacc4aa3e2398b960', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32e80d0c6d6f44c786b1053c0547de3b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-15c7c940-bccc-4288-b2ad-c1450f5336a0 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9" status: 200 len: 1492 microversion: 2.1 time: 0.304758 Apr 12 01:55:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 103/309] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:24 2023] GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 => generated 1492 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "name": "network-vif-unplugged", "status": "completed", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] Creating event network-vif-unplugged:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:55:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-02c95798-697c-41e3-b8f8-c39124aa33bb req-27ea7d73-135e-4331-b015-51e386093ab4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.100915 Apr 12 01:55:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 105/310] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:25 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2048f1cc-21c7-4d4b-b90c-bc0afaeb87ad tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-879426356", "imageRef": "", "flavorRef": "42", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "854b44b3-a8e9-4270-9161-a96a9715222e", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-2048f1cc-21c7-4d4b-b90c-bc0afaeb87ad tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] HTTP exception thrown: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. Apr 12 01:55:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2048f1cc-21c7-4d4b-b90c-bc0afaeb87ad tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:55:25 user devstack@n-api.service[72505]: request: POST /compute/v2.1/servers HTTP/1.1 Apr 12 01:55:25 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:55:25 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:55:25 user devstack@n-api.service[72505]: Connection: close Apr 12 01:55:25 user devstack@n-api.service[72505]: Content-Length: 356 Apr 12 01:55:25 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:55:25 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:55:25 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:55:25 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:55:25 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:55:25 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:55:25 user devstack@n-api.service[72505]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:55:25 user devstack@n-api.service[72505]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:55:25 user devstack@n-api.service[72505]: Apr 12 01:55:25 user devstack@n-api.service[72505]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-879426356", "imageRef": "", "flavorRef": "42", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "854b44b3-a8e9-4270-9161-a96a9715222e", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} Apr 12 01:55:25 user devstack@n-api.service[72505]: method: > Apr 12 01:55:25 user devstack@n-api.service[72505]: exception: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. Apr 12 01:55:25 user devstack@n-api.service[72505]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-879426356', 'imageRef': '', 'flavorRef': '42', 'block_device_mapping_v2': [{'boot_index': 0, 'source_type': 'volume', 'destination_type': 'volume', 'delete_on_termination': False, 'volume_id': '854b44b3-a8e9-4270-9161-a96a9715222e'}], 'networks': [{'uuid': '1d227a8b-1ad3-4bc6-a32e-4322c8f75eba'}]}}} Apr 12 01:55:25 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 785, in create Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._create_instance( Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1672, in _create_instance Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi boot_meta = block_device.get_bdm_image_metadata( Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/block_device.py", line 664, in get_bdm_image_metadata Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InvalidBDMVolumeNotBootable(id=volume_id) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InvalidBDMVolumeNotBootable: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi [Previous line repeated 11 more times] Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 867, in create Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. Apr 12 01:55:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:55:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2048f1cc-21c7-4d4b-b90c-bc0afaeb87ad tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 400 to user: Block Device 854b44b3-a8e9-4270-9161-a96a9715222e is not bootable. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:55:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2048f1cc-21c7-4d4b-b90c-bc0afaeb87ad tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 400 len: 110 microversion: 2.1 time: 0.082770 Apr 12 01:55:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 103/311] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:55:25 2023] POST /compute/v2.1/servers => generated 110 bytes in 84 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:25 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-394abc51-c195-42c1-9f8a-3a7bb2610700 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1491 microversion: 2.1 time: 0.154994 Apr 12 01:55:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 104/312] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:25 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1491 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "name": "network-vif-deleted", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] Creating event network-vif-deleted:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:55:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a25b78ca-a4ec-4cfe-a881-3e64ba541374 req-8b218904-0b66-4508-8e11-b52571ff8667 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031730 Apr 12 01:55:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 106/313] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd946144-5343-4788-b04d-ca1ed81afce9", "name": "network-vif-plugged", "status": "completed", "tag": "3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] Creating event network-vif-plugged:3aae11f1-3c64-48bf-9a1f-6e6ac2bc844e for instance fd946144-5343-4788-b04d-ca1ed81afce9 on user Apr 12 01:55:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-223f7102-ba4b-4c4b-a5a5-31db68f7dcc7 req-6cdbc263-e954-4bc2-b956-32e19f0674b3 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033858 Apr 12 01:55:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 104/314] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:27 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:34 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4544327a-6f8a-429e-bd17-9f54cc3d3e5f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.231748 Apr 12 01:55:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 105/315] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:34 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:34 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a83c8d5e-542b-48c6-9d74-32eca1ef2bb1 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.151211 Apr 12 01:55:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 107/316] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:34 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=20,nova_api:UPDATE=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:40 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lazy-loading 'services' on Instance uuid cb89325c-680a-4413-8e94-4eddfaa79c6b {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:55:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1cb737ef-08d5-4676-93a2-c192b8a0278c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments/52725f14-a644-4019-b7c6-9547d83cfb1b" status: 202 len: 0 microversion: 2.1 time: 0.320170 Apr 12 01:55:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 105/317] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:55:39 2023] DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments/52725f14-a644-4019-b7c6-9547d83cfb1b => generated 0 bytes in 322 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:55:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:42 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Going to try to stop instance {{(pid=72507) force_stop /opt/stack/nova/nova/compute/api.py:2764}} Apr 12 01:55:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7406615c-882d-486c-9b01-652c2dae7f5d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action" status: 202 len: 0 microversion: 2.1 time: 0.114303 Apr 12 01:55:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 106/318] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:55:42 2023] POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:42 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:42 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cac2c453-b9ab-4d1d-be6e-a8486cabe370 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1665 microversion: 2.1 time: 0.162044 Apr 12 01:55:42 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 108/319] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:42 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1665 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:55:42 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-unplugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] Creating event network-vif-unplugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:55:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c8a59099-57d6-486f-bdbf-f1201dee2912 req-75ca9e46-4004-4cce-8393-0394aa8c31a7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032142 Apr 12 01:55:43 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 106/320] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:42 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:55:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-54505509-da57-465b-a0e2-24f42750fbb7 req-cf59e572-1bd6-45cf-b917-46b1866cc4b0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030915 Apr 12 01:55:45 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 107/321] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:45 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:51 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:51 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:51 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:51 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-9b75d2d4-228c-4e8c-85ac-4ad19d9069fc tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 200 len: 1628 microversion: 2.49 time: 0.186717 Apr 12 01:55:51 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 109/322] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:55:50 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 1628 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=11,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-eddfd0ce-709a-41e7-89c6-5c2d079b93c3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.102694 Apr 12 01:55:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 107/323] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:55:56 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-interface => generated 283 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:56 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=14 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:56 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2acd2ee1-172a-4984-85ed-52a8e78aafb9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] HTTP exception thrown: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Request method failure captured: Apr 12 01:55:57 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 HTTP/1.1 Apr 12 01:55:57 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:55:57 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:55:57 user devstack@n-api.service[72507]: Connection: close Apr 12 01:55:57 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:55:57 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:55:57 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:55:57 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:55:57 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg46rGcCZm2W2LQiyfGvzZSDThB2IPuG0QS31eVxKy_yklGDg_-kmaerVQ3VYXF7B8aY2sH2JzsSUN2iuOggPiK_jQmCJ-UTouQc0qoR78bHRgxv4IhwuaFgqJMq-v-OZaMt8zhsjvUY7xwQHihkXfaH0zmL1mWYh_CMf_PZ2f0Bm28 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Project-Id: d0dd135bd60941dbb68b3e677c4878e9 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Tenant-Id: d0dd135bd60941dbb68b3e677c4878e9 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-User: tempest-VolumesAdminNegativeTest-312412096-project-member Apr 12 01:55:57 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:55:57 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:55:57 user devstack@n-api.service[72507]: X-User-Id: aeb7fd40946e43dc860d63bb8958d633 Apr 12 01:55:57 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesAdminNegativeTest-312412096-project-member Apr 12 01:55:57 user devstack@n-api.service[72507]: method: > Apr 12 01:55:57 user devstack@n-api.service[72507]: exception: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. Apr 12 01:55:57 user devstack@n-api.service[72507]: action_args: {'id': '2acd2ee1-172a-4984-85ed-52a8e78aafb9'} Apr 12 01:55:57 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. Apr 12 01:55:57 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:55:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Returning 404 to user: Instance 2acd2ee1-172a-4984-85ed-52a8e78aafb9 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:55:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6a1473ab-a710-4a74-894f-753944df9945 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9" status: 404 len: 111 microversion: 2.1 time: 0.136341 Apr 12 01:55:57 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 108/324] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:56 2023] GET /compute/v2.1/servers/2acd2ee1-172a-4984-85ed-52a8e78aafb9 => generated 111 bytes in 138 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=15 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:57 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=31 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:55:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] Creating event network-changed:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ServerStableDeviceRescueTest-image-1878811702"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-9da58172-4ea4-4d5f-a968-74529c0f0997 req-cba0ecb0-8495-4914-ba17-f5993e2d18e1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062179 Apr 12 01:55:57 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 110/325] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:57 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:57 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f6f66ecb-336f-4756-b926-0c7c238b6c8d tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action" status: 202 len: 0 microversion: 2.1 time: 0.847825 Apr 12 01:55:57 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 108/326] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:55:57 2023] POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action => generated 0 bytes in 850 msecs (HTTP/1.1 202) 10 headers in 449 bytes (1 switches on core 0) Apr 12 01:55:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-98324a3a-9ce8-40e3-a8ee-36c0aa383c79 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '.version_select at 0x7fb203121fc0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-98324a3a-9ce8-40e3-a8ee-36c0aa383c79 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966" status: 200 len: 1611 microversion: 2.1 time: 0.052347 Apr 12 01:55:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 109/327] 10.0.0.210 () {62 vars in 1310 bytes} [Wed Apr 12 01:55:57 2023] GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966 => generated 1611 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:59 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:55:59 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5d8b50f0-5b59-4dbb-9acc-d12257c4e061 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335" status: 204 len: 0 microversion: 2.1 time: 0.118405 Apr 12 01:55:59 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 111/328] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:55:59 2023] DELETE /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:59 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aeb7fd40946e43dc860d63bb8958d633', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0dd135bd60941dbb68b3e677c4878e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:55:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8ede2237-4b1b-42ef-9ed4-01db52ec0ccc tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335" status: 200 len: 1669 microversion: 2.1 time: 0.194835 Apr 12 01:55:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 109/329] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:55:59 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 => generated 1669 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:55:59 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "name": "network-vif-unplugged", "status": "completed", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:55:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:55:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:55:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:55:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] Creating event network-vif-unplugged:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:55:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-14ec423a-5357-438e-938a-bcc082915409 req-207e5d30-a506-4797-93f1-6f11b5ffa669 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030738 Apr 12 01:55:59 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 110/330] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:55:59 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:01 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "name": "network-vif-plugged", "status": "completed", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}, {"server_uuid": "2dbe5433-e1e5-4b85-a0f7-904c3b784335", "name": "network-vif-deleted", "tag": "f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Creating event network-vif-plugged:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:56:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] Creating event network-vif-deleted:f0adec2f-3ed3-4b7d-b435-a24c75b4cb2c for instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 on user Apr 12 01:56:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-7fb1309b-4a91-490f-8289-f1ee4aa70ab4 req-bb835895-6f71-4dc8-ac75-bc4f10817105 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.029175 Apr 12 01:56:01 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 112/331] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:01 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 9fa9bdbe-ee75-425e-bed5-049fdf908d09] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:11 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] HTTP exception thrown: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Request method failure captured: Apr 12 01:56:11 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 HTTP/1.1 Apr 12 01:56:11 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:56:11 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:56:11 user devstack@n-api.service[72505]: Connection: close Apr 12 01:56:11 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:56:11 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:56:11 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:56:11 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:56:11 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg8XBCrmSjcL9_9zotMfIbiCC6KegWlN01eMy22UTT4bRfZE55AKxrXMRtHWJJtnN5NhGJpgTuiz2QGJgy8PD21rQL8CAN0EpJOJpibxpy-QGgm9fXelfAQztwdwrU-4LZ2lfTbyC-KwfqvgZICPrzNNSZeRoOTgSy2QMwU1T248OuQ Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Project-Id: 3ce361b6fa374bca8dc0a48a53ad54f9 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesActionsTest-89585441 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesActionsTest-89585441 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Tenant-Id: 3ce361b6fa374bca8dc0a48a53ad54f9 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesActionsTest-89585441 Apr 12 01:56:11 user devstack@n-api.service[72505]: X-User: tempest-VolumesActionsTest-89585441-project-member Apr 12 01:56:11 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:56:11 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:56:11 user devstack@n-api.service[72505]: X-User-Id: 4d876b30b1724669a8d7e8bd543fa18c Apr 12 01:56:11 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesActionsTest-89585441-project-member Apr 12 01:56:11 user devstack@n-api.service[72505]: method: > Apr 12 01:56:11 user devstack@n-api.service[72505]: exception: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. Apr 12 01:56:11 user devstack@n-api.service[72505]: action_args: {'id': '9fa9bdbe-ee75-425e-bed5-049fdf908d09'} Apr 12 01:56:11 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. Apr 12 01:56:11 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:56:11 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Returning 404 to user: Instance 9fa9bdbe-ee75-425e-bed5-049fdf908d09 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:56:11 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-18742eea-83ab-41c2-9d30-dcd8685b747b tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09" status: 404 len: 111 microversion: 2.1 time: 0.045375 Apr 12 01:56:11 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 110/332] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:11 2023] GET /compute/v2.1/servers/9fa9bdbe-ee75-425e-bed5-049fdf908d09 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:56:11 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:11 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1665338189", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "a0d34b37-6e08-4f2e-a7df-43c3b7d8080f"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:11 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] validate_networks() for [('a0d34b37-6e08-4f2e-a7df-43c3b7d8080f', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Getting quotas for project 3ce361b6fa374bca8dc0a48a53ad54f9. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Getting quotas for user 4d876b30b1724669a8d7e8bd543fa18c and project 3ce361b6fa374bca8dc0a48a53ad54f9. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:56:12 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:56:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6ee88c0d-4cbd-4ca3-b669-7d20484791db tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.806812 Apr 12 01:56:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 111/333] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:56:11 2023] POST /compute/v2.1/servers => generated 378 bytes in 808 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:56:12 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=10,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:12 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:12 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-06ca906e-8572-4214-8002-94b771a4537d tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:12 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-06ca906e-8572-4214-8002-94b771a4537d tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:12 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-06ca906e-8572-4214-8002-94b771a4537d tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:12 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-06ca906e-8572-4214-8002-94b771a4537d tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c" status: 200 len: 1208 microversion: 2.1 time: 0.080492 Apr 12 01:56:12 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 113/334] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:12 2023] GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c => generated 1208 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:56:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] Creating event network-changed:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:56:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-fb3f45f1-f5e1-4e2c-b636-74c0bb82ff6a req-f29391a6-455a-4fc9-9805-a17aeace012d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029979 Apr 12 01:56:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 111/335] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:14 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] [instance: fd946144-5343-4788-b04d-ca1ed81afce9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] HTTP exception thrown: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Request method failure captured: Apr 12 01:56:15 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 HTTP/1.1 Apr 12 01:56:15 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:56:15 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:56:15 user devstack@n-api.service[72507]: Connection: close Apr 12 01:56:15 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:56:15 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:56:15 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:56:15 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:56:15 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg8XzkRvzc9vX7066cFRf6X2fMVeJgE_4aCh1PvwuN-LiIBa8lssdVhrc4D7KfqoxCDVEN9VZiiHYH_SoJkvHET3Ggbv1n_mH9KQaPABDCFeItDCeOCLemXvzlup0ZME4eHEiKTuXb8mnipz0By2k6JFwzRY5QVVUCo9zL0JzXTqIUk Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Project-Id: 32e80d0c6d6f44c786b1053c0547de3b Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesActionsTest-621357222 Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/32e80d0c6d6f44c786b1053c0547de3b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/32e80d0c6d6f44c786b1053c0547de3b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/32e80d0c6d6f44c786b1053c0547de3b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesActionsTest-621357222 Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Tenant-Id: 32e80d0c6d6f44c786b1053c0547de3b Apr 12 01:56:15 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesActionsTest-621357222 Apr 12 01:56:15 user devstack@n-api.service[72507]: X-User: tempest-VolumesActionsTest-621357222-project-member Apr 12 01:56:15 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:56:15 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:56:15 user devstack@n-api.service[72507]: X-User-Id: a92cd627a1c04cedacc4aa3e2398b960 Apr 12 01:56:15 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesActionsTest-621357222-project-member Apr 12 01:56:15 user devstack@n-api.service[72507]: method: > Apr 12 01:56:15 user devstack@n-api.service[72507]: exception: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. Apr 12 01:56:15 user devstack@n-api.service[72507]: action_args: {'id': 'fd946144-5343-4788-b04d-ca1ed81afce9'} Apr 12 01:56:15 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. Apr 12 01:56:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] Returning 404 to user: Instance fd946144-5343-4788-b04d-ca1ed81afce9 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:56:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e4e4aecf-83b6-411d-a96a-026db68399c7 tempest-VolumesActionsTest-621357222 tempest-VolumesActionsTest-621357222-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9" status: 404 len: 111 microversion: 2.1 time: 0.036730 Apr 12 01:56:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 112/336] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:15 2023] GET /compute/v2.1/servers/fd946144-5343-4788-b04d-ca1ed81afce9 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:56:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "name": "network-vif-plugged", "status": "completed", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] Creating event network-vif-plugged:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:56:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c6604adf-919f-407d-adc7-59d2137f5ee5 req-5350795f-f7bb-4117-91e3-42921391664b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029650 Apr 12 01:56:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 114/337] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:18 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "name": "network-vif-plugged", "status": "completed", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] Creating event network-vif-plugged:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:56:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-b6b494ae-6906-4979-816a-b99e97c57575 req-750b0506-f873-44cd-aad1-ac682ac41df5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031261 Apr 12 01:56:18 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 112/338] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7eca1c3b-60dd-4608-bd1b-41d0f08aef56"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] HTTP exception thrown: Cannot 'attach_volume' instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 while it is in vm_state rescued Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Request method failure captured: Apr 12 01:56:24 user devstack@n-api.service[72507]: request: POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/os-volume_attachments HTTP/1.1 Apr 12 01:56:24 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:56:24 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:56:24 user devstack@n-api.service[72507]: Connection: close Apr 12 01:56:24 user devstack@n-api.service[72507]: Content-Length: 74 Apr 12 01:56:24 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:56:24 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:56:24 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:56:24 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg44PjYOKvn626k4LU-GdnrYvKUw0JPm986bERmaP8HWBpoYllBF-zT9F33eDRWVPNLXGnPnqSXLqxvxPxbpnKT-DsqjWHFcOAkLCgQVhgVNvj2O847gDgRAlXz1fgsLlBnnV14mxHU4brxBUpZaN6c1uNs-kqCIbkSyeXxWQGBSWz0 Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Project-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Tenant-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 01:56:24 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 01:56:24 user devstack@n-api.service[72507]: X-User: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 01:56:24 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:56:24 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:56:24 user devstack@n-api.service[72507]: X-User-Id: 889189da060e4b54a48ce9a013e04a3d Apr 12 01:56:24 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 01:56:24 user devstack@n-api.service[72507]: Apr 12 01:56:24 user devstack@n-api.service[72507]: {"volumeAttachment": {"volumeId": "7eca1c3b-60dd-4608-bd1b-41d0f08aef56"}} Apr 12 01:56:24 user devstack@n-api.service[72507]: method: > Apr 12 01:56:24 user devstack@n-api.service[72507]: exception: Cannot 'attach_volume' instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 while it is in vm_state rescued Apr 12 01:56:24 user devstack@n-api.service[72507]: action_args: {'server_id': 'fdbd8add-ac0e-42ce-9b34-5878c6bec208', 'body': {'volumeAttachment': {'volumeId': '7eca1c3b-60dd-4608-bd1b-41d0f08aef56'}}} Apr 12 01:56:24 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 in vm_state rescued. Cannot attach_volume while the instance is in this state. Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 while it is in vm_state rescued Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 while it is in vm_state rescued Apr 12 01:56:24 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:56:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Returning 409 to user: Cannot 'attach_volume' instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 while it is in vm_state rescued {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:56:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-675b2650-796c-460f-a8ab-bd90338027af tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.066000 Apr 12 01:56:24 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 113/339] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:56:24 2023] POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/os-volume_attachments => generated 152 bytes in 67 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b2ccaae9-b0da-4ea4-bc93-61526f45ce68 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/action" status: 202 len: 0 microversion: 2.1 time: 0.117080 Apr 12 01:56:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 115/340] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:56:24 2023] POST /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208/action => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:24 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:24 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-00ec1a8b-507b-4c2c-83e5-fe922aef4313 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1497 microversion: 2.1 time: 0.211656 Apr 12 01:56:24 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 113/341] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:24 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1497 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-unplugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:25 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] Creating event network-vif-unplugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:56:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-60108523-9f9c-4457-9ff2-d418b8bdd16e tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1491 microversion: 2.1 time: 0.178987 Apr 12 01:56:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 114/342] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:25 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1491 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-44ad3bfd-4ec8-41cb-88d1-b591012bc4d0 req-4a412f0a-6b92-4914-ad60-532eba4b9261 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.040872 Apr 12 01:56:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 116/343] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:25 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}, {"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:56:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:56:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-21a78c1f-ad34-4992-9249-901d93e2df47 req-2849389f-56b5-4b6b-8760-a208a5308912 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.040902 Apr 12 01:56:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 114/344] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:27 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 01:56:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-deea4b83-6a15-4b76-af7d-b3689eba02c3 req-689e62d5-eef6-49b8-877b-c7982919b482 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030029 Apr 12 01:56:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 115/345] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:29 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:33 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0e13bf19-f306-4c38-aed7-77aec72129d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1642 microversion: 2.1 time: 0.174030 Apr 12 01:56:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 117/346] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:32 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1642 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:34 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lazy-loading 'services' on Instance uuid c712f2db-72cb-4136-a719-56057d82fff6 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:56:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-78f59ee2-f0b2-44d5-ae4e-eb080f642470 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments/333bfdb2-c5a0-443a-bcc1-c203e669c896" status: 202 len: 0 microversion: 2.1 time: 0.198484 Apr 12 01:56:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 115/347] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:56:34 2023] DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments/333bfdb2-c5a0-443a-bcc1-c203e669c896 => generated 0 bytes in 201 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-966e72dc-e762-438b-8c06-70d4a3bac3d8 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052509 Apr 12 01:56:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 116/348] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:56:34 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-afcdc334-c2a4-46cb-829e-4f1fb1bfa2e3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '.version_select at 0x7fb2031539a0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=20,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-afcdc334-c2a4-46cb-829e-4f1fb1bfa2e3 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966" status: 200 len: 1725 microversion: 2.1 time: 0.054345 Apr 12 01:56:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 118/349] 10.0.0.210 () {62 vars in 1310 bytes} [Wed Apr 12 01:56:48 2023] GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966 => generated 1725 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=12 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ad1c6e33-6cda-440f-a640-9a1883dd40f0 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '.version_select at 0x7fb2031c5ea0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ad1c6e33-6cda-440f-a640-9a1883dd40f0 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966" status: 200 len: 1725 microversion: 2.1 time: 0.062390 Apr 12 01:56:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 116/350] 10.0.0.210 () {62 vars in 1310 bytes} [Wed Apr 12 01:56:48 2023] GET /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966 => generated 1725 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=23,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=22 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-199f1a4c-323b-457f-9514-84d31f2d2c4e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1672 microversion: 2.1 time: 0.156079 Apr 12 01:56:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 117/351] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:48 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1672 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] [instance: 2dbe5433-e1e5-4b85-a0f7-904c3b784335] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] HTTP exception thrown: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Request method failure captured: Apr 12 01:56:49 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 HTTP/1.1 Apr 12 01:56:49 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:56:49 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:56:49 user devstack@n-api.service[72506]: Connection: close Apr 12 01:56:49 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:56:49 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:56:49 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:56:49 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:56:49 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg46rGcCZm2W2LQiyfGvzZSDThB2IPuG0QS31eVxKy_yklGDg_-kmaerVQ3VYXF7B8aY2sH2JzsSUN2iuOggPiK_jQmCJ-UTouQc0qoR78bHRgxv4IhwuaFgqJMq-v-OZaMt8zhsjvUY7xwQHihkXfaH0zmL1mWYh_CMf_PZ2f0Bm28 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Project-Id: d0dd135bd60941dbb68b3e677c4878e9 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Project-Name: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/d0dd135bd60941dbb68b3e677c4878e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Tenant: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Tenant-Id: d0dd135bd60941dbb68b3e677c4878e9 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-312412096 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-User: tempest-VolumesAdminNegativeTest-312412096-project-member Apr 12 01:56:49 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:56:49 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:56:49 user devstack@n-api.service[72506]: X-User-Id: aeb7fd40946e43dc860d63bb8958d633 Apr 12 01:56:49 user devstack@n-api.service[72506]: X-User-Name: tempest-VolumesAdminNegativeTest-312412096-project-member Apr 12 01:56:49 user devstack@n-api.service[72506]: method: > Apr 12 01:56:49 user devstack@n-api.service[72506]: exception: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. Apr 12 01:56:49 user devstack@n-api.service[72506]: action_args: {'id': '2dbe5433-e1e5-4b85-a0f7-904c3b784335'} Apr 12 01:56:49 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. Apr 12 01:56:49 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:56:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Returning 404 to user: Instance 2dbe5433-e1e5-4b85-a0f7-904c3b784335 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:56:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-22695dc0-bcab-4617-9a3e-ed7d4502f437 tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335" status: 404 len: 111 microversion: 2.1 time: 0.032135 Apr 12 01:56:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 119/352] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:49 2023] GET /compute/v2.1/servers/2dbe5433-e1e5-4b85-a0f7-904c3b784335 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:56:49 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c2bfd24f-f505-483b-985e-3af4e7bcd06b tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] Calling method '.version_select at 0x7fb2031c4ca0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:49 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c2bfd24f-f505-483b-985e-3af4e7bcd06b tempest-VolumesAdminNegativeTest-312412096 tempest-VolumesAdminNegativeTest-312412096-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-599771826" status: 202 len: 0 microversion: 2.1 time: 0.032896 Apr 12 01:56:49 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 117/353] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 01:56:49 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-599771826 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:51 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6df77b6a-b1e1-489c-9190-19155c3f52f8 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface" status: 200 len: 283 microversion: 2.49 time: 0.091250 Apr 12 01:56:51 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 118/354] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 01:56:51 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface => generated 283 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:56:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] Creating event network-changed:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:56:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f1012d4c-e1ba-48aa-8a2b-af01ea07661b req-38b73126-7c1c-45af-8027-423c2e472c13 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033826 Apr 12 01:56:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 120/355] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:52 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:52 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e79746e8-0c31-46c5-93cd-995f8a0c53a5 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 200 len: 1628 microversion: 2.49 time: 0.191285 Apr 12 01:56:52 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 118/356] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:56:52 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 1628 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Action: 'create', calling method: >, body: {"interfaceAttachment": {"net_id": "2dceb67c-b24a-4d04-b054-93f03677d195", "tag": "nic-tag"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:55 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:55 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:55 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] Creating event network-changed:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:56:55 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-682d0627-499a-4ae4-9217-1c1c26409596 req-ce3d5fc1-5899-4e25-aa03-79113aa2bb9d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034260 Apr 12 01:56:55 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 121/357] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:55 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1646401490", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:56 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-49183713-33e2-465c-abae-dfebcddce082 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface" status: 200 len: 280 microversion: 2.49 time: 3.830919 Apr 12 01:56:56 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 119/358] 10.0.0.210 () {66 vars in 1414 bytes} [Wed Apr 12 01:56:52 2023] POST /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface => generated 280 bytes in 3832 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0b915cc2-ce61-4b9d-832d-412e056f70ad", "tag": "volume-tag"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:56 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] validate_networks() for None {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-11cc6d73-9863-4d55-93ca-e59d42886719 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments" status: 200 len: 194 microversion: 2.49 time: 0.274664 Apr 12 01:56:57 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 122/359] 10.0.0.210 () {66 vars in 1441 bytes} [Wed Apr 12 01:56:56 2023] POST /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments => generated 194 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:56:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] Creating event network-vif-plugged:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:56:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-022be81f-4bb5-4eb4-a152-f0e612d6b16f req-67e1cb56-b8be-4caa-b696-9ddf5e2b180a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039799 Apr 12 01:56:57 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 120/360] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:57 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Getting quotas for project fd34c505d1bc40a394bfbd2ae2edd9fb. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Getting quotas for user ac9601d0624f4a19863473f6b14f2bf3 and project fd34c505d1bc40a394bfbd2ae2edd9fb. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:56:57 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:56:57 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-36baf0e7-1af9-4bf2-864d-2ab2acc6c603 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 1.177078 Apr 12 01:56:57 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 119/361] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:56:56 2023] POST /compute/v2.1/servers => generated 378 bytes in 1182 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:56:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-21985dfa-d35a-4c98-b44a-bc63f67df85e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:56:58 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-21985dfa-d35a-4c98-b44a-bc63f67df85e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:56:58 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-21985dfa-d35a-4c98-b44a-bc63f67df85e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac9601d0624f4a19863473f6b14f2bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd34c505d1bc40a394bfbd2ae2edd9fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:56:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-21985dfa-d35a-4c98-b44a-bc63f67df85e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285" status: 200 len: 1209 microversion: 2.1 time: 0.063617 Apr 12 01:56:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 123/362] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:56:57 2023] GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 => generated 1209 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:56:59 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:56:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:56:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:56:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:56:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] Creating event network-vif-plugged:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:56:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-aaf3562f-2a46-4a2e-9cde-ced64a260fba req-bd30690e-37a9-42a0-a3ad-ef54d1ef7b14 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032326 Apr 12 01:56:59 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 121/363] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:56:59 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:01 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] Creating event network-changed:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:57:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-bdc69fb4-5b91-4755-8d90-9bae9cfbdaa7 req-e2324280-e6eb-4ba3-aac1-7d9d28439fd7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053396 Apr 12 01:57:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 120/364] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:01 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:02 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-46f9e56b-0ddd-4a50-9b3b-32510923690c tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c" status: 200 len: 1486 microversion: 2.1 time: 0.175433 Apr 12 01:57:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 124/365] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:02 2023] GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c => generated 1486 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "name": "network-vif-plugged", "status": "completed", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] Creating event network-vif-plugged:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:57:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-4b70313d-af04-481d-bb78-043e2e45ecba req-3c6120c9-e0a1-4d66-a510-76f1a48d96ad service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033720 Apr 12 01:57:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 122/366] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:03 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:05 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lazy-loading 'services' on Instance uuid 5b7cbacb-f776-4e9a-b333-ce9c345dc525 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:57:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "name": "network-vif-plugged", "status": "completed", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] Creating event network-vif-plugged:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:57:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f22c8efb-0967-4dfb-8937-0451870dbf68 req-501baadd-0541-4c1c-bfa1-7cd0011be468 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037260 Apr 12 01:57:05 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 125/367] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a40d6f49-8736-4537-a6c8-87bf605bb95e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments/0b915cc2-ce61-4b9d-832d-412e056f70ad" status: 202 len: 0 microversion: 2.49 time: 0.179400 Apr 12 01:57:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 121/368] 10.0.0.210 () {64 vars in 1534 bytes} [Wed Apr 12 01:57:05 2023] DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments/0b915cc2-ce61-4b9d-832d-412e056f70ad => generated 0 bytes in 182 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-fc1ec44d-dd14-4084-938f-553c730b2149 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface/bb6652b3-90cf-4337-a94b-ca982c4397f5" status: 202 len: 0 microversion: 2.49 time: 0.052661 Apr 12 01:57:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 123/369] 10.0.0.210 () {64 vars in 1507 bytes} [Wed Apr 12 01:57:07 2023] DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-interface/bb6652b3-90cf-4337-a94b-ca982c4397f5 => generated 0 bytes in 55 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-1a627d80-3634-4bdf-bf0d-23beee35ea46 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149" status: 200 len: 474 microversion: 2.49 time: 0.332922 Apr 12 01:57:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 126/370] 10.0.0.210 () {64 vars in 1537 bytes} [Wed Apr 12 01:57:07 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149 => generated 474 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-unplugged", "status": "completed", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] Creating event network-vif-unplugged:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:57:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a98170c8-ff6c-4dbf-bd7d-4fd1034b8642 req-69100215-ce4d-43e4-a969-18d6c6c28234 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.052083 Apr 12 01:57:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 122/371] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:08 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}, {"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-deleted", "tag": "bb6652b3-90cf-4337-a94b-ca982c4397f5"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Creating event network-vif-plugged:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:57:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] Creating event network-vif-deleted:bb6652b3-90cf-4337-a94b-ca982c4397f5 for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:57:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-39742746-ffcc-468e-8d06-57d8c56115cb req-5ffb2ced-d65c-4ece-891e-e68f2ee3b92e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.034452 Apr 12 01:57:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 124/372] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:10 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:15 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a358eb86-ff3f-474b-a4de-99a3cde03473 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1503 microversion: 2.1 time: 0.176621 Apr 12 01:57:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 127/373] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:14 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1503 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1188952548", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "c12e9557-d89a-46e7-8232-823d5738fa26"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:19 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] validate_networks() for [('c12e9557-d89a-46e7-8232-823d5738fa26', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Getting quotas for project 1dbc688a91f547c0adbf929d0f714d51. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Getting quotas for user cb64ad9101674c1a92d41dfd57ca8e15 and project 1dbc688a91f547c0adbf929d0f714d51. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:57:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:57:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b483a6d4-e138-4d50-a6bd-fdbb902606d5 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.576543 Apr 12 01:57:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 123/374] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:57:19 2023] POST /compute/v2.1/servers => generated 378 bytes in 578 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Apr 12 01:57:20 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=18 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:20 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=32,nova_cell1:INSERT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:20 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d15c4ab1-93bb-4cd1-bc64-40111d65fe58 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:20 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d15c4ab1-93bb-4cd1-bc64-40111d65fe58 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:20 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-d15c4ab1-93bb-4cd1-bc64-40111d65fe58 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d15c4ab1-93bb-4cd1-bc64-40111d65fe58 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca" status: 200 len: 1213 microversion: 2.1 time: 0.083304 Apr 12 01:57:20 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 125/375] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:20 2023] GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca => generated 1213 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=21 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:23 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] Creating event network-changed:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:57:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-94ff49c8-a78c-43be-9e45-11661ef40aba req-100ce5d8-b86a-4277-8c48-7ab23cc52218 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037373 Apr 12 01:57:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 128/376] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:23 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4c966157-d811-47ff-a539-c769b32feeda tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049350 Apr 12 01:57:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 124/377] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:57:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-50504505-4805-4d03-9d80-d774ecd3ad75 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1677 microversion: 2.1 time: 0.174204 Apr 12 01:57:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 126/378] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1677 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4fa2d6f3-2a1c-4124-b468-d3bd8cf0fee0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1677 microversion: 2.1 time: 0.239887 Apr 12 01:57:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 129/379] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1677 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "name": "network-vif-plugged", "status": "completed", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] Creating event network-vif-plugged:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:57:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-c3a82d7b-ab56-4152-b778-631a72e219c0 req-e6f3c4df-6922-4bd7-92ad-f0ee52477247 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028658 Apr 12 01:57:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 127/380] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:25 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3f622745-b912-49ab-bf00-5e5bed74d4ba tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1677 microversion: 2.1 time: 0.158268 Apr 12 01:57:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 125/381] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1677 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cdd6e005-ec21-4507-ad41-ee3a01e8ab99 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-828991273", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "destination_type": "local"}, {"source_type": "volume", "uuid": "0530376d-7656-4909-8d55-2ef8287ecdf7", "destination_type": "invalid"}], "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cdd6e005-ec21-4507-ad41-ee3a01e8ab99 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:57:27 user devstack@n-api.service[72506]: request: POST /compute/v2.1/servers HTTP/1.1 Apr 12 01:57:27 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:57:27 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:57:27 user devstack@n-api.service[72506]: Connection: close Apr 12 01:57:27 user devstack@n-api.service[72506]: Content-Length: 493 Apr 12 01:57:27 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:57:27 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:57:27 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:57:27 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:57:27 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:57:27 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:57:27 user devstack@n-api.service[72506]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:57:27 user devstack@n-api.service[72506]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:57:27 user devstack@n-api.service[72506]: Apr 12 01:57:27 user devstack@n-api.service[72506]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-828991273", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "destination_type": "local"}, {"source_type": "volume", "uuid": "0530376d-7656-4909-8d55-2ef8287ecdf7", "destination_type": "invalid"}], "networks": [{"uuid": "1d227a8b-1ad3-4bc6-a32e-4322c8f75eba"}]}} Apr 12 01:57:27 user devstack@n-api.service[72506]: method: > Apr 12 01:57:27 user devstack@n-api.service[72506]: exception: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Apr 12 01:57:27 user devstack@n-api.service[72506]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-828991273', 'imageRef': '16f93c82-82f4-4843-b632-b6b6fe0b6c16', 'flavorRef': '42', 'block_device_mapping_v2': [{'source_type': 'image', 'delete_on_termination': True, 'boot_index': 0, 'uuid': '16f93c82-82f4-4843-b632-b6b6fe0b6c16', 'destination_type': 'local'}, {'source_type': 'volume', 'uuid': '0530376d-7656-4909-8d55-2ef8287ecdf7', 'destination_type': 'invalid'}], 'networks': [{'uuid': '1d227a8b-1ad3-4bc6-a32e-4322c8f75eba'}]}}} Apr 12 01:57:27 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 314, in validate Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise error Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['local', 'volume'] Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['server']['properties']['block_device_mapping_v2']['items']['properties']['destination_type']: Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi {'enum': ['local', 'volume'], 'type': 'string'} Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi On instance['server']['block_device_mapping_v2'][1]: Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi 'invalid' Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Apr 12 01:57:27 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:57:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cdd6e005-ec21-4507-ad41-ee3a01e8ab99 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 400 to user: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:57:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cdd6e005-ec21-4507-ad41-ee3a01e8ab99 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 400 len: 155 microversion: 2.1 time: 0.012959 Apr 12 01:57:27 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 130/382] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:57:27 2023] POST /compute/v2.1/servers => generated 155 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:27 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:27 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3a26cd00-a75a-4e7a-b017-00fa8f4df386 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1491 microversion: 2.1 time: 0.221835 Apr 12 01:57:27 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 128/383] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:27 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1491 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "name": "network-vif-plugged", "status": "completed", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] Creating event network-vif-plugged:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:57:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-9b6eedd9-7504-42c1-8d6e-edd483a255fd req-083db390-0006-4fb8-bd9e-ea890b4191e5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033005 Apr 12 01:57:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 126/384] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:27 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Going to try to start instance {{(pid=72506) start /opt/stack/nova/nova/compute/api.py:2787}} Apr 12 01:57:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c92635c6-b1bb-475f-95df-aec9664c414b tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action" status: 202 len: 0 microversion: 2.1 time: 0.069303 Apr 12 01:57:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 131/385] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:57:33 2023] POST /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/action => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:33 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a0878ab0-2844-4b98-aaed-dee8768a13e2 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1651 microversion: 2.1 time: 0.163715 Apr 12 01:57:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 129/386] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:33 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1651 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:36 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:57:36 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-883f9495-dc79-4d31-942c-bdbab6177616 req-7f0c47d6-d347-411b-8e13-e83d593e63e3 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041305 Apr 12 01:57:36 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 127/387] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:36 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:57:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-3e1445c7-eca2-404c-81dc-a23dad43d32c req-f7f8f479-b664-4cfc-95ef-753e1850b762 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029329 Apr 12 01:57:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 132/388] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:38 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=20 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac9601d0624f4a19863473f6b14f2bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd34c505d1bc40a394bfbd2ae2edd9fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e5a5cda1-86f5-4332-b0c0-d1f994fd0195 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285" status: 200 len: 1455 microversion: 2.1 time: 0.148957 Apr 12 01:57:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 130/389] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:48 2023] GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 => generated 1455 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=18 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:48 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-67178d89-c41d-4ed1-9556-f8a091cf772c tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1672 microversion: 2.1 time: 0.177799 Apr 12 01:57:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 128/390] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:48 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1672 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=7 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cf91b105-163e-4c1d-a936-6939ccb3db3f"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e74cb50a-432b-4e45-8b69-1ec56b60fdd2 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.301333 Apr 12 01:57:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 133/391] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:57:50 2023] POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments => generated 194 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'action', calling method: >, body: {"rescue": {"rescue_image_ref": "37146b86-9ec3-440f-95de-7bf8ac2dd966"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lazy-loading 'flavor' on Instance uuid 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:57:57 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2e452ff3-e327-4826-a795-9c480ec1286f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action" status: 200 len: 29 microversion: 2.1 time: 0.203539 Apr 12 01:57:57 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 131/392] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:57:57 2023] POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action => generated 29 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:57 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-46ae15a4-3c56-4aa7-80a0-b207a33828fd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1724 microversion: 2.1 time: 0.291138 Apr 12 01:57:57 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 129/393] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:57:57 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1724 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:57 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-797be062-95d4-4f00-926a-e743bad1897b tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149" status: 200 len: 503 microversion: 2.49 time: 0.061993 Apr 12 01:57:57 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 134/394] 10.0.0.210 () {64 vars in 1537 bytes} [Wed Apr 12 01:57:57 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149 => generated 503 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4eba53b1-0924-411d-92db-136fd677f052 tempest-TaggedAttachmentsTest-96993965 tempest-TaggedAttachmentsTest-96993965-project-admin] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149" status: 200 len: 503 microversion: 2.49 time: 0.094905 Apr 12 01:57:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 132/395] 10.0.0.210 () {64 vars in 1537 bytes} [Wed Apr 12 01:57:58 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-instance-actions/req-fc1ec44d-dd14-4084-938f-553c730b2149 => generated 503 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-unplugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-cf395387-df54-44ed-b30c-678a9a948e20 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments" status: 200 len: 25 microversion: 2.49 time: 0.050617 Apr 12 01:57:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 130/396] 10.0.0.210 () {64 vars in 1420 bytes} [Wed Apr 12 01:57:58 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] Creating event network-vif-unplugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:57:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-daba229b-9b82-4837-b35b-8c7f5bb65a27 req-7b197acb-038a-44ed-b567-6c7e4c38bf7d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.028687 Apr 12 01:57:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 135/397] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:58 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:57:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-84f795d1-1f67-4ee4-ad00-96e76a5cc389 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 204 len: 0 microversion: 2.49 time: 0.267390 Apr 12 01:57:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 133/398] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:57:58 2023] DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-unplugged", "status": "completed", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:57:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '43819af3565d4eff92ce1a51f86df4f2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3443faad6b314920a85b8491603e30ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:57:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d52eb294-f06c-4004-9763-64de82b9193e tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 200 len: 1752 microversion: 2.49 time: 0.298317 Apr 12 01:57:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 131/399] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:57:58 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 1752 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:57:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:57:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] Creating event network-vif-unplugged:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:57:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-d35801a0-4365-4f4e-b790-9c8e499911de req-687274e6-8f92-4c6a-b92e-b6a6f48ab9e2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037779 Apr 12 01:57:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 136/400] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:57:58 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:58:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-cff599ae-c8dc-4184-b747-8da8df57a886 req-39219a44-2e1b-406d-bd40-8abb35fc8a8a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033779 Apr 12 01:58:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 134/401] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:00 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-plugged", "status": "completed", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] Creating event network-vif-plugged:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:58:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-09514ea4-03e0-4d13-8f0b-33343696c36d req-9e10a123-6cc8-45f9-9758-418136d4de73 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033975 Apr 12 01:58:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 132/402] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:00 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b7cbacb-f776-4e9a-b333-ce9c345dc525", "name": "network-vif-deleted", "tag": "0e649e0b-d93b-473c-a765-b0173cea1a6d"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Creating event network-vif-deleted:0e649e0b-d93b-473c-a765-b0173cea1a6d for instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 on user Apr 12 01:58:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:58:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-baef3814-c75a-494a-8a50-77cf45431ef7 req-2342c6a2-73ed-4a5c-b021-0a5e47b6838b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.038051 Apr 12 01:58:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 137/403] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:02 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:58:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e6d3ee6c-7a7f-4acc-bec6-660003aacff2 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c" status: 204 len: 0 microversion: 2.1 time: 0.116685 Apr 12 01:58:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 135/404] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:58:03 2023] DELETE /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:03 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4d876b30b1724669a8d7e8bd543fa18c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3ce361b6fa374bca8dc0a48a53ad54f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9a956e53-b5f8-4d4a-8dce-7eb6aa2ba115 tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c" status: 200 len: 1492 microversion: 2.1 time: 0.169125 Apr 12 01:58:03 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 133/405] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:03 2023] GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c => generated 1492 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:03 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "name": "network-vif-unplugged", "status": "completed", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:03 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] Creating event network-vif-unplugged:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:58:03 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-e8aa05ad-143c-4f45-a996-39d92587409b req-fc9238ce-f4bc-48b2-8d1c-7b7b3c95ad11 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.028860 Apr 12 01:58:03 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 138/406] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:03 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:04 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:58:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8c1ba4cc-09cf-48c8-95e9-fd036f2f64e8 req-2d2f4e86-0b6a-4afa-9fe1-b0180a792e56 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032955 Apr 12 01:58:04 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 136/407] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:04 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "name": "network-vif-plugged", "status": "completed", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] Creating event network-vif-plugged:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:58:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-1d148b9e-9ca8-4931-a0ec-332f2f4873ec req-91801ab0-5b8f-4e8d-b645-ec11355c8891 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038981 Apr 12 01:58:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 134/408] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:06 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c1ed52-fe5b-4e57-854a-36476bdb215c", "name": "network-vif-deleted", "tag": "4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:06 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:06 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] Creating event network-vif-deleted:4adf6f7b-2ca4-41b3-b8e4-85a2b15f6dec for instance 39c1ed52-fe5b-4e57-854a-36476bdb215c on user Apr 12 01:58:06 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-73513515-a2e9-460f-a7d4-380164e70ec0 req-8aa546aa-06e7-483b-8f31-734b67c96d26 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046614 Apr 12 01:58:06 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 139/409] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:06 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:10 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f84b550b-85fb-4463-9642-eab12be28b40 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca" status: 200 len: 1496 microversion: 2.1 time: 0.153986 Apr 12 01:58:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 137/410] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:10 2023] GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca => generated 1496 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e8dc8e4f-421f-46bf-abc8-d0250c89d7b3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: .version_select at 0x7fb2031c4d30>, body: {"keypair": {"name": "tempest-keypair-2119544665", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDIbKc6xSzjYSTxxI8C/IAZs7b7MlyezBiUXhjYxXfwthgDrpaDZJYY7wb6RFJBJGBUJGdDeu+0PCYgm+2/EOL1dCVnyqQ1ADD+YRouh7yBs349raBu7ykBoZaVMgsZIuA=="}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=34 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:16 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-e8dc8e4f-421f-46bf-abc8-d0250c89d7b3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for project 8fed0886b2404f41a659dc6356771e9c. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:58:16 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-e8dc8e4f-421f-46bf-abc8-d0250c89d7b3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for user 889189da060e4b54a48ce9a013e04a3d and project 8fed0886b2404f41a659dc6356771e9c. Resources: {'key_pairs'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:58:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e8dc8e4f-421f-46bf-abc8-d0250c89d7b3 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.071359 Apr 12 01:58:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 135/411] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:58:16 2023] POST /compute/v2.1/os-keypairs => generated 386 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=14 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=29,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1373752537", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "babcad8e-dca2-47b4-80af-a9e0cede0853"}], "security_groups": [{"name": "tempest-securitygroup--1375604231"}], "key_name": "tempest-keypair-2119544665", "user_data": "}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:18 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] validate_networks() for [('babcad8e-dca2-47b4-80af-a9e0cede0853', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Getting quotas for user 889189da060e4b54a48ce9a013e04a3d and project 8fed0886b2404f41a659dc6356771e9c. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:58:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:58:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4377791a-50c3-4bc0-b97f-ddd11601e5cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 0.891223 Apr 12 01:58:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 140/412] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:58:18 2023] POST /compute/v2.1/servers => generated 407 bytes in 893 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:58:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0a0b7f93-8ddb-412b-bd0b-3e11c028686b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0a0b7f93-8ddb-412b-bd0b-3e11c028686b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:19 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:19 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0a0b7f93-8ddb-412b-bd0b-3e11c028686b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0a0b7f93-8ddb-412b-bd0b-3e11c028686b tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1240 microversion: 2.1 time: 0.073309 Apr 12 01:58:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 138/413] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:19 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1240 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:58:22 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:22 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] Creating event network-changed:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 01:58:22 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-9aa54fb4-96eb-421d-bae8-ba6a4264053e req-ad481e13-0d9e-43e4-b90c-c457c72c0bff service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029949 Apr 12 01:58:22 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 136/414] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:22 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:23 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-af08f478-e868-4073-86b2-ba3b6f1ac8b3 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1655 microversion: 2.1 time: 0.230507 Apr 12 01:58:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 141/415] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:23 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1655 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:24 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 01:58:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-26d24e72-847b-46e6-bb6e-35c97802b257 req-31d68faa-ca05-461f-85c7-82c60337d892 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043603 Apr 12 01:58:24 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 139/416] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:24 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:26 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 01:58:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-15006d1a-a9ae-4ff5-b5b6-5badc6bcda84 req-b28b5147-34ea-4d1a-b418-e854507dc6af service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031877 Apr 12 01:58:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 137/417] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:26 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:27 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c0a7fd93-79f2-47c6-9412-41e78fa20453 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1677 microversion: 2.1 time: 0.163240 Apr 12 01:58:27 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 142/418] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:27 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1677 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7c40d13a-f773-494b-be5c-b559dfc5686a"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:27 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-87726f8b-1462-4b19-9049-97bc06199ad4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.283072 Apr 12 01:58:27 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 140/419] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:58:27 2023] POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments => generated 194 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] HTTP exception thrown: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:58:32 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 HTTP/1.1 Apr 12 01:58:32 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:58:32 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:58:32 user devstack@n-api.service[72505]: Connection: close Apr 12 01:58:32 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:58:32 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:58:32 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:58:32 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:58:32 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:58:32 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:58:32 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:58:32 user devstack@n-api.service[72505]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:58:32 user devstack@n-api.service[72505]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:58:32 user devstack@n-api.service[72505]: method: > Apr 12 01:58:32 user devstack@n-api.service[72505]: exception: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72505]: action_args: {'id': '2f32ace1-1adc-4b75-a1cc-daac2e724846'} Apr 12 01:58:32 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 404 to user: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6350dcb8-0f35-4276-ba70-c2acd74bd1cf tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 404 len: 111 microversion: 2.1 time: 0.030422 Apr 12 01:58:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 138/420] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:58:32 2023] DELETE /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: 2f32ace1-1adc-4b75-a1cc-daac2e724846] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] HTTP exception thrown: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:58:32 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 HTTP/1.1 Apr 12 01:58:32 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:58:32 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:58:32 user devstack@n-api.service[72506]: Connection: close Apr 12 01:58:32 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:58:32 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:58:32 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:58:32 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:58:32 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:58:32 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:58:32 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:58:32 user devstack@n-api.service[72506]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:58:32 user devstack@n-api.service[72506]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:58:32 user devstack@n-api.service[72506]: method: > Apr 12 01:58:32 user devstack@n-api.service[72506]: exception: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72506]: action_args: {'id': '2f32ace1-1adc-4b75-a1cc-daac2e724846'} Apr 12 01:58:32 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. Apr 12 01:58:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 404 to user: Instance 2f32ace1-1adc-4b75-a1cc-daac2e724846 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-efa14717-8c2a-4282-90fb-e857797591e5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846" status: 404 len: 111 microversion: 2.1 time: 0.040200 Apr 12 01:58:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 143/421] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:32 2023] GET /compute/v2.1/servers/2f32ace1-1adc-4b75-a1cc-daac2e724846 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:58:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2de62f61-c072-4c68-bb64-67c9fc16afc5 tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 204 len: 0 microversion: 2.1 time: 0.120981 Apr 12 01:58:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 141/422] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:58:32 2023] DELETE /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '728c534ef1f54c50ad2b619b44c3b9c7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '021f9aee2b7849ef8fcd67d6d6522d27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-cfb35f9d-7f48-4b4e-805d-d8c61ff73c8b tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 200 len: 1497 microversion: 2.1 time: 0.189064 Apr 12 01:58:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 139/423] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:32 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 1497 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "name": "network-vif-unplugged", "status": "completed", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] Creating event network-vif-unplugged:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:58:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-0c7915cd-5944-48f3-8cce-3159b7c0b4fe req-5d812448-3585-4b5f-bdf2-f65f4114e8c5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032167 Apr 12 01:58:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 144/424] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:32 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "name": "network-vif-deleted", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] Creating event network-vif-deleted:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:58:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-04a78901-6e81-4fdc-8233-91a277cf6a29 req-0353dd67-2807-40df-8ed3-ccc100eb0b74 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030075 Apr 12 01:58:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 142/425] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:33 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dfe43143-ac3d-4aed-9a6e-5ec27fb609e4", "name": "network-vif-plugged", "status": "completed", "tag": "d776e917-e4e5-4688-b138-ffb1eebbb61f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] Creating event network-vif-plugged:d776e917-e4e5-4688-b138-ffb1eebbb61f for instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 on user Apr 12 01:58:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-29a53d0f-195b-4a46-8cbf-a4bc5d8cb06e req-a92b7513-bcc1-4fb5-bc28-0dd896650497 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028527 Apr 12 01:58:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 140/426] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "84"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Old instance type m1.nano, new instance type m1.micro {{(pid=72506) resize /opt/stack/nova/nova/compute/api.py:4223}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:58:35 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:58:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-09800ba9-a41b-439c-b27b-c97441ffb1dc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action" status: 202 len: 0 microversion: 2.1 time: 0.263176 Apr 12 01:58:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 145/427] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:58:35 2023] POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action => generated 0 bytes in 265 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:35 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5e2766bd-b523-48db-96ce-038f2c7e5b7f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1732 microversion: 2.1 time: 0.190471 Apr 12 01:58:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 143/428] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:35 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1732 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-unplugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Creating event network-vif-unplugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:58:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72505) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 01:58:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8ea30855-67d6-47c3-aa42-94354d5b0d02 req-eca5cb63-68ea-4b59-bf79-09ca9551668a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045269 Apr 12 01:58:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 141/429] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:38 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:40 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:58:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72506) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 01:58:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-bd2612ef-a3fc-4abe-972d-a228dad6fbd5 req-3fe5241d-6b6f-4ce3-b922-dcf53c3bad83 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048866 Apr 12 01:58:40 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 146/430] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:40 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=36,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:47 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:47 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-55769b3f-192f-4ed5-a6fb-97ce5c69593e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1704 microversion: 2.1 time: 0.175056 Apr 12 01:58:47 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 144/431] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:47 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1704 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=23,nova_api:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=18 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:58:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7b076fa0-9f1a-4c8d-9fc2-8c05985c431e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285" status: 204 len: 0 microversion: 2.1 time: 0.141182 Apr 12 01:58:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 142/432] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:58:48 2023] DELETE /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:58:48 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac9601d0624f4a19863473f6b14f2bf3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd34c505d1bc40a394bfbd2ae2edd9fb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:58:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5a0e0db9-8978-4bd3-b2c4-5d3ea2cdda5e tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285" status: 200 len: 1461 microversion: 2.1 time: 0.232901 Apr 12 01:58:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 147/433] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:48 2023] GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 => generated 1461 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:58:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "name": "network-vif-unplugged", "status": "completed", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] Creating event network-vif-unplugged:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:58:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b613408e-7aff-4aa4-aca7-0e9d5f39f2c3 req-054a30bf-8596-402e-884b-050afba90f9b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042780 Apr 12 01:58:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 145/434] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:48 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] HTTP exception thrown: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Request method failure captured: Apr 12 01:58:48 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 HTTP/1.1 Apr 12 01:58:48 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:58:48 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:58:48 user devstack@n-api.service[72505]: Connection: close Apr 12 01:58:48 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:58:48 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:58:48 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:58:48 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:58:48 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg9ojp7_SZ7DfJCmE0qGkDJbFhTpusTUd5QQDDg2aO1Oge7W4YUyevktvBxp7zR5bAZENxvgDIbOjTdJmZn8RsbuJsQGL0G5o0TCyaXrMD5zBRrj5b3v0j2tR8GYaxMJnegW7Bq2dC2rhM889ljzOq3LlC5tS0o4r9DQZpZbIolZbrM Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.49 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Project-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3443faad6b314920a85b8491603e30ac"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Tenant-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:48 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-User: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:48 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:58:48 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:58:48 user devstack@n-api.service[72505]: X-User-Id: 43819af3565d4eff92ce1a51f86df4f2 Apr 12 01:58:48 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:48 user devstack@n-api.service[72505]: method: > Apr 12 01:58:48 user devstack@n-api.service[72505]: exception: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:48 user devstack@n-api.service[72505]: action_args: {'id': '5b7cbacb-f776-4e9a-b333-ce9c345dc525'} Apr 12 01:58:48 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:48 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Returning 404 to user: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-13181258-d23c-42ad-b11c-9ddedb189087 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 404 len: 111 microversion: 2.49 time: 0.039139 Apr 12 01:58:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 143/435] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:58:48 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:58:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-202ddd22-4c7a-4f2e-a021-67414f85d7e3 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '.version_select at 0x7fb20308b910>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-202ddd22-4c7a-4f2e-a021-67414f85d7e3 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1476535441" status: 204 len: 0 microversion: 2.49 time: 0.021606 Apr 12 01:58:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 148/436] 10.0.0.210 () {64 vars in 1339 bytes} [Wed Apr 12 01:58:48 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1476535441 => generated 0 bytes in 23 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 01:58:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:58:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72507) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 01:58:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-bc563623-8ed3-4aa6-a289-963c971b191f req-f89fa241-9194-47a6-b64b-237523aede56 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058963 Apr 12 01:58:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 146/437] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:49 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] HTTP exception thrown: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Request method failure captured: Apr 12 01:58:50 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 HTTP/1.1 Apr 12 01:58:50 user devstack@n-api.service[72505]: Accept: application/json Apr 12 01:58:50 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 01:58:50 user devstack@n-api.service[72505]: Connection: close Apr 12 01:58:50 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 01:58:50 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 01:58:50 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 01:58:50 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 01:58:50 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg9ojp7_SZ7DfJCmE0qGkDJbFhTpusTUd5QQDDg2aO1Oge7W4YUyevktvBxp7zR5bAZENxvgDIbOjTdJmZn8RsbuJsQGL0G5o0TCyaXrMD5zBRrj5b3v0j2tR8GYaxMJnegW7Bq2dC2rhM889ljzOq3LlC5tS0o4r9DQZpZbIolZbrM Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.49 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Project-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Project-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3443faad6b314920a85b8491603e30ac"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Tenant: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Tenant-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:50 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-User: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:50 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 01:58:50 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 01:58:50 user devstack@n-api.service[72505]: X-User-Id: 43819af3565d4eff92ce1a51f86df4f2 Apr 12 01:58:50 user devstack@n-api.service[72505]: X-User-Name: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:50 user devstack@n-api.service[72505]: method: > Apr 12 01:58:50 user devstack@n-api.service[72505]: exception: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72505]: action_args: {'id': '5b7cbacb-f776-4e9a-b333-ce9c345dc525'} Apr 12 01:58:50 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Returning 404 to user: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1f3b8aa3-45fa-40fd-84db-c66fb7d940f6 tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 404 len: 111 microversion: 2.49 time: 0.028708 Apr 12 01:58:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 144/438] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 01:58:50 2023] DELETE /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] [instance: 5b7cbacb-f776-4e9a-b333-ce9c345dc525] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] HTTP exception thrown: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Request method failure captured: Apr 12 01:58:50 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 HTTP/1.1 Apr 12 01:58:50 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:58:50 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:58:50 user devstack@n-api.service[72506]: Connection: close Apr 12 01:58:50 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:58:50 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:58:50 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:58:50 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:58:50 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg9ojp7_SZ7DfJCmE0qGkDJbFhTpusTUd5QQDDg2aO1Oge7W4YUyevktvBxp7zR5bAZENxvgDIbOjTdJmZn8RsbuJsQGL0G5o0TCyaXrMD5zBRrj5b3v0j2tR8GYaxMJnegW7Bq2dC2rhM889ljzOq3LlC5tS0o4r9DQZpZbIolZbrM Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Openstack-Nova-Api-Version: 2.49 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Project-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Project-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3443faad6b314920a85b8491603e30ac"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3443faad6b314920a85b8491603e30ac"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Tenant: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Tenant-Id: 3443faad6b314920a85b8491603e30ac Apr 12 01:58:50 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TaggedAttachmentsTest-1804071739 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-User: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:50 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:58:50 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:58:50 user devstack@n-api.service[72506]: X-User-Id: 43819af3565d4eff92ce1a51f86df4f2 Apr 12 01:58:50 user devstack@n-api.service[72506]: X-User-Name: tempest-TaggedAttachmentsTest-1804071739-project-member Apr 12 01:58:50 user devstack@n-api.service[72506]: method: > Apr 12 01:58:50 user devstack@n-api.service[72506]: exception: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72506]: action_args: {'id': '5b7cbacb-f776-4e9a-b333-ce9c345dc525'} Apr 12 01:58:50 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. Apr 12 01:58:50 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:58:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] Returning 404 to user: Instance 5b7cbacb-f776-4e9a-b333-ce9c345dc525 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-be86ead6-b91b-4b80-b3ea-868bd905b15f tempest-TaggedAttachmentsTest-1804071739 tempest-TaggedAttachmentsTest-1804071739-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525" status: 404 len: 111 microversion: 2.49 time: 0.032926 Apr 12 01:58:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 149/439] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 01:58:50 2023] GET /compute/v2.1/servers/5b7cbacb-f776-4e9a-b333-ce9c345dc525 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 01:58:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "name": "network-vif-plugged", "status": "completed", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] Creating event network-vif-plugged:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:58:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-774c41ed-93fa-41a6-8d9e-4d143ad2d05e req-21bcc9df-2b3a-43cc-92ab-8679be326f69 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036650 Apr 12 01:58:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 147/440] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:50 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 01:58:51 user devstack@n-api.service[72505]: DEBUG nova.compute.api [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72505) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 01:58:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-d9b13786-b4c1-454b-b13b-890f1c7bc655 req-31ac9037-9dcd-40c3-8bbe-68f28f165476 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038457 Apr 12 01:58:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 145/441] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:51 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3827c534-7f4a-4f22-bf88-91e8897c6285", "name": "network-vif-deleted", "tag": "b0d250e6-f783-4891-aeed-e2204b3b2662"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:58:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] Creating event network-vif-deleted:b0d250e6-f783-4891-aeed-e2204b3b2662 for instance 3827c534-7f4a-4f22-bf88-91e8897c6285 on user Apr 12 01:58:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-95a004a8-2e11-42b6-a361-fc38cd1ccd27 req-65ef84ae-7099-4fa6-8d5d-35d69c7b1499 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040231 Apr 12 01:58:53 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 150/442] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:58:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] [instance: 39c1ed52-fe5b-4e57-854a-36476bdb215c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:58:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] HTTP exception thrown: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Request method failure captured: Apr 12 01:58:53 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c HTTP/1.1 Apr 12 01:58:53 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:58:53 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:58:53 user devstack@n-api.service[72507]: Connection: close Apr 12 01:58:53 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:58:53 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:58:53 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:58:53 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:58:53 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg8XBCrmSjcL9_9zotMfIbiCC6KegWlN01eMy22UTT4bRfZE55AKxrXMRtHWJJtnN5NhGJpgTuiz2QGJgy8PD21rQL8CAN0EpJOJpibxpy-QGgm9fXelfAQztwdwrU-4LZ2lfTbyC-KwfqvgZICPrzNNSZeRoOTgSy2QMwU1T248OuQ Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Project-Id: 3ce361b6fa374bca8dc0a48a53ad54f9 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesActionsTest-89585441 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/3ce361b6fa374bca8dc0a48a53ad54f9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesActionsTest-89585441 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Tenant-Id: 3ce361b6fa374bca8dc0a48a53ad54f9 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesActionsTest-89585441 Apr 12 01:58:53 user devstack@n-api.service[72507]: X-User: tempest-VolumesActionsTest-89585441-project-member Apr 12 01:58:53 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:58:53 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:58:53 user devstack@n-api.service[72507]: X-User-Id: 4d876b30b1724669a8d7e8bd543fa18c Apr 12 01:58:53 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesActionsTest-89585441-project-member Apr 12 01:58:53 user devstack@n-api.service[72507]: method: > Apr 12 01:58:53 user devstack@n-api.service[72507]: exception: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. Apr 12 01:58:53 user devstack@n-api.service[72507]: action_args: {'id': '39c1ed52-fe5b-4e57-854a-36476bdb215c'} Apr 12 01:58:53 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. Apr 12 01:58:53 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:58:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] Returning 404 to user: Instance 39c1ed52-fe5b-4e57-854a-36476bdb215c could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:58:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-092fc099-02eb-41f6-baf0-29838d8a07cf tempest-VolumesActionsTest-89585441 tempest-VolumesActionsTest-89585441-project-member] 10.0.0.210 "GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c" status: 404 len: 111 microversion: 2.1 time: 0.144592 Apr 12 01:58:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 148/443] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:58:53 2023] GET /compute/v2.1/servers/39c1ed52-fe5b-4e57-854a-36476bdb215c => generated 111 bytes in 146 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:09 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-05b8757f-913a-40f0-9cb7-00140e25d252 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1554 microversion: 2.1 time: 0.186189 Apr 12 01:59:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 146/444] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:09 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1554 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=9 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "16ef6179-f08e-48ff-80c4-d5301b48eecf", "device": "/dev/vdb"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4069ba3b-cdb2-480e-8567-4d592f4d7520 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "POST /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.331872 Apr 12 01:59:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 151/445] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 01:59:10 2023] POST /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca/os-volume_attachments => generated 194 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] [instance: dfe43143-ac3d-4aed-9a6e-5ec27fb609e4] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=16 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] HTTP exception thrown: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Request method failure captured: Apr 12 01:59:22 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 HTTP/1.1 Apr 12 01:59:22 user devstack@n-api.service[72507]: Accept: application/json Apr 12 01:59:22 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 01:59:22 user devstack@n-api.service[72507]: Connection: close Apr 12 01:59:22 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 01:59:22 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 01:59:22 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 01:59:22 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 01:59:22 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg48ClUt6TMHYiVzOacs-ukqfiYV_sivb0NfGDmTCVLPuQOYB7PXX_gl0T6XWmuicSFQIwJz_RiRQjy6ciyYpUwb88SC3NFqRYTkbJbZL76LRhvdeaKDULB9ehpEGQkabxLD1QPn_Aakm9V94cFhIjkIMDi_a4fYwfKFC4OhdLbLY3U Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Project-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/021f9aee2b7849ef8fcd67d6d6522d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Tenant: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Tenant-Id: 021f9aee2b7849ef8fcd67d6d6522d27 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServersNegativeTestJSON-292036935 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-User: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:59:22 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 01:59:22 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 01:59:22 user devstack@n-api.service[72507]: X-User-Id: 728c534ef1f54c50ad2b619b44c3b9c7 Apr 12 01:59:22 user devstack@n-api.service[72507]: X-User-Name: tempest-ServersNegativeTestJSON-292036935-project-member Apr 12 01:59:22 user devstack@n-api.service[72507]: method: > Apr 12 01:59:22 user devstack@n-api.service[72507]: exception: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. Apr 12 01:59:22 user devstack@n-api.service[72507]: action_args: {'id': 'dfe43143-ac3d-4aed-9a6e-5ec27fb609e4'} Apr 12 01:59:22 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. Apr 12 01:59:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 01:59:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] Returning 404 to user: Instance dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:59:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2c1a9262-800e-418e-9394-bda6b93cc18c tempest-ServersNegativeTestJSON-292036935 tempest-ServersNegativeTestJSON-292036935-project-member] 10.0.0.210 "GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4" status: 404 len: 111 microversion: 2.1 time: 0.037674 Apr 12 01:59:22 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 149/446] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:22 2023] GET /compute/v2.1/servers/dfe43143-ac3d-4aed-9a6e-5ec27fb609e4 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 01:59:23 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:23 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9a9e7b95-cd27-495f-8815-0cbc5e9c1b5a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.083737 Apr 12 01:59:24 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 147/447] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 01:59:23 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b/os-volume_attachments => generated 25 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:59:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-932b4a81-21eb-4d0d-aef7-0eaf38e5e8d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 204 len: 0 microversion: 2.1 time: 0.167235 Apr 12 01:59:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 152/448] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:59:24 2023] DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:24 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a49a9bf7-817e-4ef9-bf35-50d0d71889e0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 200 len: 1661 microversion: 2.1 time: 0.394077 Apr 12 01:59:24 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 150/449] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:24 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 1661 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-unplugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] Creating event network-vif-unplugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:59:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-ab49d243-abf0-415f-b7c2-a7e312555e0b req-507128fa-af50-40e3-a037-13061d529a3f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046305 Apr 12 01:59:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 148/450] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:25 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:26 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-deleted", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:26 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-3f94588f-0cae-42a5-be24-359a2dc7b3cf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1731 microversion: 2.1 time: 0.359525 Apr 12 01:59:26 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] Creating event network-vif-deleted:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:59:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 153/451] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1731 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:26 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3ea52263-212d-4ae0-9c9a-893eae76a339 req-4347e827-caa9-4387-896f-5adfc3c4c7cb service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048796 Apr 12 01:59:26 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 151/452] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:26 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb89325c-680a-4413-8e94-4eddfaa79c6b", "name": "network-vif-plugged", "status": "completed", "tag": "ec368fc5-6d05-4bab-8460-cf9f1e024195"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] Creating event network-vif-plugged:ec368fc5-6d05-4bab-8460-cf9f1e024195 for instance cb89325c-680a-4413-8e94-4eddfaa79c6b on user Apr 12 01:59:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-233e2365-1f18-4794-8e0e-eb4b9c01481f req-fdc01536-cfec-4990-81e6-43914070d552 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037564 Apr 12 01:59:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 149/453] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:27 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lazy-loading 'services' on Instance uuid 77f15843-a3a9-4943-a763-0901cd84d7ca {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 01:59:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-3456cf07-4898-4923-9512-77641472c3e4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca/os-volume_attachments/16ef6179-f08e-48ff-80c4-d5301b48eecf" status: 202 len: 0 microversion: 2.1 time: 0.170066 Apr 12 01:59:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 154/454] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 01:59:29 2023] DELETE /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca/os-volume_attachments/16ef6179-f08e-48ff-80c4-d5301b48eecf => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 01:59:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-33c81f30-c9ad-4eff-90d3-38ec50a15e76 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca" status: 204 len: 0 microversion: 2.1 time: 0.126119 Apr 12 01:59:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 152/455] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 01:59:29 2023] DELETE /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:29 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7d39f24c-fac4-4875-8c4e-c1f38c557a9d tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca" status: 200 len: 1548 microversion: 2.1 time: 0.312160 Apr 12 01:59:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 150/456] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:29 2023] GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca => generated 1548 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "name": "network-vif-unplugged", "status": "completed", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] Creating event network-vif-unplugged:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:59:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-43b2e4af-ce47-4827-8328-7bca604c3732 req-0bbf58cf-b360-4ba7-bca6-f6f5c016e29b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035147 Apr 12 01:59:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 155/457] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:32 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "name": "network-vif-deleted", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] Creating event network-vif-deleted:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:59:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-2e0dd4af-0cc1-4856-b8d5-4e043be5e533 req-741f42b6-a023-461c-81e2-2e5e28a812d8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044005 Apr 12 01:59:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 153/458] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77f15843-a3a9-4943-a763-0901cd84d7ca", "name": "network-vif-plugged", "status": "completed", "tag": "00d413e3-0f38-46cf-baf4-c5b61bbcf104"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] Creating event network-vif-plugged:00d413e3-0f38-46cf-baf4-c5b61bbcf104 for instance 77f15843-a3a9-4943-a763-0901cd84d7ca on user Apr 12 01:59:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8bfdc8fb-2747-4489-9001-e99280a433c6 req-2764fee1-5300-4a7b-940d-2929ab756c1e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038866 Apr 12 01:59:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 151/459] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-ca9d345b-f3d8-4de1-95e4-54844bb7f9f8 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: .version_select at 0x7fb202fd7b50>, body: {"keypair": {"name": "tempest-keypair-97228139", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPyQPgNnY+n5pmjFnlz3V3O4w81ZzobntB8dh+2yVFNBNym/eGJJo7urnH/zX2dzy5CA7558+VofCPs+9Vde/YZ1+ziQKJzVrqQuhHVBk0QzvwcemmNSiwyqvFvAu2VGhg=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-ca9d345b-f3d8-4de1-95e4-54844bb7f9f8 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:59:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-ca9d345b-f3d8-4de1-95e4-54844bb7f9f8 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:59:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-ca9d345b-f3d8-4de1-95e4-54844bb7f9f8 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.283397 Apr 12 01:59:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 156/460] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:59:35 2023] POST /compute/v2.1/os-keypairs => generated 384 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:37 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fc223392-b11f-46d8-b7f4-05c0e2d888a8 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Action: 'create', calling method: .version_select at 0x7fb2030b5ab0>, body: {"keypair": {"name": "tempest-TestShelveInstance-269156000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEzJEkyxrgGdFi9xp4HEHo6h+HMI/sf0D1BEcLp1PlBC0aEIXZkpEAZB7dj1l2oKNIXETzcME23LJ7KETYvsoSN/f9VetiXUt+tFF8xxyxwD7zg/09NP8sJhNpL272IgEg=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:37 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-fc223392-b11f-46d8-b7f4-05c0e2d888a8 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Getting quotas for project 97353e0d7e084a1095d6ae46416d5660. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:59:37 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-fc223392-b11f-46d8-b7f4-05c0e2d888a8 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Getting quotas for user 5108c5bde1e249a2934f3e47496b1027 and project 97353e0d7e084a1095d6ae46416d5660. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:59:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-fc223392-b11f-46d8-b7f4-05c0e2d888a8 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.154344 Apr 12 01:59:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 154/461] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 01:59:37 2023] POST /compute/v2.1/os-keypairs => generated 396 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-376857066", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "0875dfb2-68e9-4a37-8295-74f038cfba9a"}], "security_groups": [{"name": "tempest-securitygroup--1776598213"}], "key_name": "tempest-keypair-97228139", "user_data": "}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] validate_networks() for [('0875dfb2-68e9-4a37-8295-74f038cfba9a', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] [instance: 3827c534-7f4a-4f22-bf88-91e8897c6285] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] HTTP exception thrown: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Request method failure captured: Apr 12 01:59:38 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 HTTP/1.1 Apr 12 01:59:38 user devstack@n-api.service[72506]: Accept: application/json Apr 12 01:59:38 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 01:59:38 user devstack@n-api.service[72506]: Connection: close Apr 12 01:59:38 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 01:59:38 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 01:59:38 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 01:59:38 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 01:59:38 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg_naIDWJm99zb3ssAL51jHdQybmr0sus97ULQmogF5HSa9KgV6WH6q4kE_7lCCyQwDASSLfL1Tq4ltlTN6F5IrlN2FNH9QdkOgnE44gFI_mRgCPiFzcB_w3JH84Lj8bY9tVXesWF_Og5WZSFjsTGHM1ZSapSvJFyb2xpeAAvuXH2OI Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Project-Id: fd34c505d1bc40a394bfbd2ae2edd9fb Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Project-Name: tempest-VolumesNegativeTest-523576533 Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/fd34c505d1bc40a394bfbd2ae2edd9fb"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/fd34c505d1bc40a394bfbd2ae2edd9fb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/fd34c505d1bc40a394bfbd2ae2edd9fb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Tenant: tempest-VolumesNegativeTest-523576533 Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Tenant-Id: fd34c505d1bc40a394bfbd2ae2edd9fb Apr 12 01:59:38 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-VolumesNegativeTest-523576533 Apr 12 01:59:38 user devstack@n-api.service[72506]: X-User: tempest-VolumesNegativeTest-523576533-project-member Apr 12 01:59:38 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 01:59:38 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 01:59:38 user devstack@n-api.service[72506]: X-User-Id: ac9601d0624f4a19863473f6b14f2bf3 Apr 12 01:59:38 user devstack@n-api.service[72506]: X-User-Name: tempest-VolumesNegativeTest-523576533-project-member Apr 12 01:59:38 user devstack@n-api.service[72506]: method: > Apr 12 01:59:38 user devstack@n-api.service[72506]: exception: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. Apr 12 01:59:38 user devstack@n-api.service[72506]: action_args: {'id': '3827c534-7f4a-4f22-bf88-91e8897c6285'} Apr 12 01:59:38 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. Apr 12 01:59:38 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 01:59:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] Returning 404 to user: Instance 3827c534-7f4a-4f22-bf88-91e8897c6285 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 01:59:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-2286c1ef-bb39-494b-b1c0-5be73e8be745 tempest-VolumesNegativeTest-523576533 tempest-VolumesNegativeTest-523576533-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285" status: 404 len: 111 microversion: 2.1 time: 0.028879 Apr 12 01:59:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 157/462] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:38 2023] GET /compute/v2.1/servers/3827c534-7f4a-4f22-bf88-91e8897c6285 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:59:38 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:59:39 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:59:39 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:59:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4e840369-9510-416b-a48d-d64881b79adf tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.852606 Apr 12 01:59:39 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 152/463] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:59:38 2023] POST /compute/v2.1/servers => generated 400 bytes in 854 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:59:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b058d26e-a145-4aa3-b913-7b8875ef834d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:39 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-b058d26e-a145-4aa3-b913-7b8875ef834d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:39 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-b058d26e-a145-4aa3-b913-7b8875ef834d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b058d26e-a145-4aa3-b913-7b8875ef834d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 200 len: 1233 microversion: 2.1 time: 0.185734 Apr 12 01:59:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 155/464] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:39 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 1233 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] Creating event network-changed:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 01:59:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-81d371f9-4466-47da-a369-a0cc909b1719 req-7c711d3a-b1c4-443c-ba2b-5f2a7d88224e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028168 Apr 12 01:59:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 158/465] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:41 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-662581423", "imageRef": "", "flavorRef": "42", "key_name": "tempest-TestShelveInstance-269156000", "security_groups": [{"name": "tempest-secgroup-smoke-36189640"}], "networks": [{"uuid": "6773c41d-3492-43c6-b74a-688f2d7b9fc5"}], "block_device_mapping_v2": [{"uuid": "11d89fd1-2c63-48c0-8341-1af121bc4cc7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] validate_networks() for [('6773c41d-3492-43c6-b74a-688f2d7b9fc5', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Getting quotas for project 97353e0d7e084a1095d6ae46416d5660. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Getting quotas for user 5108c5bde1e249a2934f3e47496b1027 and project 97353e0d7e084a1095d6ae46416d5660. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 01:59:42 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='11d89fd1-2c63-48c0-8341-1af121bc4cc7',volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 01:59:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b5dc48ab-71e7-438e-8d77-c6909fd6c246 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.784615 Apr 12 01:59:42 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 153/466] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 01:59:42 2023] POST /compute/v2.1/servers => generated 402 bytes in 787 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 01:59:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-db5d43ac-8dff-4da6-9b95-7a293954faa7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:42 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-db5d43ac-8dff-4da6-9b95-7a293954faa7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:42 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-db5d43ac-8dff-4da6-9b95-7a293954faa7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 01:59:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-db5d43ac-8dff-4da6-9b95-7a293954faa7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1082 microversion: 2.1 time: 0.076367 Apr 12 01:59:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 156/467] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 01:59:42 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1082 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 01:59:44 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "name": "network-vif-plugged", "status": "completed", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] Creating event network-vif-plugged:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 01:59:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-676adec5-936d-4c34-818b-e5f968ea2d6b req-89f3a742-ba71-4c1e-a6f5-b361fc82bd9e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031282 Apr 12 01:59:44 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 159/468] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:44 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "name": "network-vif-plugged", "status": "completed", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] Creating event network-vif-plugged:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 01:59:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-168841d1-8f59-4c3c-9000-126000c4f393 req-d00e4c59-42e3-4a2c-a5cb-5d976511529e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037484 Apr 12 01:59:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 154/469] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:46 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:47 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=27,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-100a920d-d10a-49ef-969a-37ea33639fba tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action" status: 202 len: 0 microversion: 2.1 time: 0.247038 Apr 12 01:59:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 157/470] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 01:59:47 2023] POST /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/action => generated 0 bytes in 249 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Apr 12 01:59:48 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 01:59:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] Creating event network-changed:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-6853cdf7-9a35-4d5e-bca0-0e78e9184f5a req-d2b6aedc-8a91-4a3a-9350-ab21b9f08d05 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034900 Apr 12 01:59:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 160/471] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:48 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-dfe33b4a-1960-4ba1-92fd-b3d094388c0f tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.181562 Apr 12 01:59:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 155/472] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 01:59:48 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-interface => generated 283 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-unplugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"name": "network-changed", "server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"name": "network-changed", "server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Creating event network-vif-unplugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Creating event network-changed:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Creating event network-changed:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-5258a66b-91e9-42d0-a843-fa084a035037 req-c6c603f9-4e7f-4a05-b94f-bc95c4e40d67 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 861 microversion: 2.1 time: 0.039212 Apr 12 01:59:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 158/473] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:50 2023] POST /compute/v2.1/os-server-external-events => generated 861 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-unplugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}, {"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Creating event network-vif-unplugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a1de25a2-ee46-4a12-a555-37b3ad85f2f7 req-4704818b-2d74-4835-b6cb-0602cfd2f399 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 698 microversion: 2.1 time: 0.039913 Apr 12 01:59:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 161/474] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:52 2023] POST /compute/v2.1/os-server-external-events => generated 698 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=12 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:54 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=29 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 01:59:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-6b88437a-0637-4da5-a750-0edd7bb99797 req-4331277a-95b8-44f6-b320-d211f4bae88d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062576 Apr 12 01:59:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 156/475] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:54 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:54 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 01:59:54 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-77e0fb7e-ae5e-4a1e-8098-bde6b9abeec7 req-c7d5179b-3852-4a35-8016-5dc967548643 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045924 Apr 12 01:59:54 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 159/476] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:54 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 01:59:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 01:59:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 01:59:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8821393e-a0ab-41e0-b5b0-97d9645f398d req-13bfcdda-3d7a-4227-beab-024ff07a59ef service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034908 Apr 12 01:59:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 162/477] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 01:59:56 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=46,nova_api:INSERT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e400cb2b-e770-4021-bae5-22225d07bcd6 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.152782 Apr 12 02:00:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 157/478] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:00:09 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-interface => generated 282 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=14,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f6f462d6-7d7c-4843-97c3-d0ded790997f"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] Creating event network-changed:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:00:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e664f1bc-0e24-4273-8291-29351539d226 req-0b7de438-68bf-4e7e-8773-19b4d2f5f5f7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.316849 Apr 12 02:00:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 160/479] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:10 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d4416006-f402-4c20-a06a-62390eaee894 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.459170 Apr 12 02:00:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 163/480] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:00:10 2023] POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments => generated 194 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:00:14 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b HTTP/1.1 Apr 12 02:00:14 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:00:14 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:00:14 user devstack@n-api.service[72505]: Connection: close Apr 12 02:00:14 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:00:14 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:00:14 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:00:14 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:00:14 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:00:14 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:00:14 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:00:14 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:00:14 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:00:14 user devstack@n-api.service[72505]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:00:14 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:00:14 user devstack@n-api.service[72505]: method: > Apr 12 02:00:14 user devstack@n-api.service[72505]: exception: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:00:14 user devstack@n-api.service[72505]: action_args: {'id': 'cb89325c-680a-4413-8e94-4eddfaa79c6b'} Apr 12 02:00:14 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:00:14 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:00:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e40cedca-c25f-4f41-8983-4713d14054a0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 404 len: 111 microversion: 2.1 time: 0.036105 Apr 12 02:00:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 158/481] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:14 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:00:14 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8467fd95-242d-478e-8fab-249ed002ae62 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '.version_select at 0x7fb20349c550>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:14 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-8467fd95-242d-478e-8fab-249ed002ae62 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-663896141" status: 202 len: 0 microversion: 2.1 time: 0.024907 Apr 12 02:00:14 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 161/482] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:00:14 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-663896141 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:00:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-69351572-ae68-43d2-b28d-33f81325b67d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: .version_select at 0x7fb202e44c10>, body: {"keypair": {"name": "tempest-keypair-155530966", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO9PbnZpthoEPBbgAsjmQsY6RgSWKMwste0tDXW/CGuop5MpgsjfMlEApyCMQgwuzCsJ9Gr8VF6fKEHWivLB+lL3P2qNsPHCK4NwqoKfCUSZQWI/Rq+KTRt1Nfy0sVaPxw=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:15 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-69351572-ae68-43d2-b28d-33f81325b67d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:00:15 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-69351572-ae68-43d2-b28d-33f81325b67d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for user d2ca4855c3e9461cb27c88469086d7d8 and project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:00:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-69351572-ae68-43d2-b28d-33f81325b67d tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.049095 Apr 12 02:00:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 164/483] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:00:15 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1553393100", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "10203cb1-b210-4779-b45b-c0822f9562cb"}], "security_groups": [{"name": "tempest-securitygroup--3149629"}], "key_name": "tempest-keypair-155530966", "user_data": "}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] validate_networks() for [('10203cb1-b210-4779-b45b-c0822f9562cb', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lazy-loading 'flavor' on Instance uuid 84b4902c-24d9-4e96-b51f-e93e913aa681 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:00:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7136b801-9c89-44da-8e64-06244f8eb689 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/action" status: 200 len: 32 microversion: 2.1 time: 0.152643 Apr 12 02:00:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 162/484] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:00:17 2023] POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/action => generated 32 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Getting quotas for user d2ca4855c3e9461cb27c88469086d7d8 and project c34bd636a2a54b249dca9ce9027ebf4b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:00:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:00:17 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-2d5a1ecc-95ee-4d64-bfbc-3c04433dec18 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1725 microversion: 2.1 time: 0.226891 Apr 12 02:00:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-836eb926-0035-42e4-8d2d-864a2b746f7a tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 397 microversion: 2.1 time: 0.666090 Apr 12 02:00:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 165/485] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:17 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1725 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 159/486] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:00:17 2023] POST /compute/v2.1/servers => generated 397 bytes in 669 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a0ac0514-583e-4f9d-adce-81c54d3ae09c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a0ac0514-583e-4f9d-adce-81c54d3ae09c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:17 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-a0ac0514-583e-4f9d-adce-81c54d3ae09c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a0ac0514-583e-4f9d-adce-81c54d3ae09c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 200 len: 1231 microversion: 2.1 time: 0.067218 Apr 12 02:00:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 163/487] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:17 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 1231 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-unplugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] Creating event network-vif-unplugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:00:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-5466d399-823b-4cdb-b4be-ee8dbe5c3925 req-ca8c6acc-c6d7-4c6b-84fd-d717f353dcda service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037899 Apr 12 02:00:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 166/488] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:18 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: 77f15843-a3a9-4943-a763-0901cd84d7ca] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] HTTP exception thrown: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Request method failure captured: Apr 12 02:00:19 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca HTTP/1.1 Apr 12 02:00:19 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:00:19 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:00:19 user devstack@n-api.service[72505]: Connection: close Apr 12 02:00:19 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:00:19 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:00:19 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:00:19 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:00:19 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg_7Gp7NjeUBhd0X0pUaopW-UaAw4pgOJoXNZ04FgC8Oo6fg8K_1Yd-2SM7IC8fO0oO74CEV4gVk_wJsvjCqLLZWW-VoyWbSBWFNLjgAG7qt1tQOw7eObsk3ZNI56cbByTskLXt0OJGRFO4NH2xshxvfG7RoCISud0gKS6NTdr-lwFE Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Project-Id: 1dbc688a91f547c0adbf929d0f714d51 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Tenant-Id: 1dbc688a91f547c0adbf929d0f714d51 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-User: tempest-VolumesSnapshotTestJSON-1019440197-project-member Apr 12 02:00:19 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:00:19 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:00:19 user devstack@n-api.service[72505]: X-User-Id: cb64ad9101674c1a92d41dfd57ca8e15 Apr 12 02:00:19 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesSnapshotTestJSON-1019440197-project-member Apr 12 02:00:19 user devstack@n-api.service[72505]: method: > Apr 12 02:00:19 user devstack@n-api.service[72505]: exception: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. Apr 12 02:00:19 user devstack@n-api.service[72505]: action_args: {'id': '77f15843-a3a9-4943-a763-0901cd84d7ca'} Apr 12 02:00:19 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. Apr 12 02:00:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:00:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Returning 404 to user: Instance 77f15843-a3a9-4943-a763-0901cd84d7ca could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:00:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a4d40f7e-b18b-4cb7-87cc-faf023dcb4f4 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca" status: 404 len: 111 microversion: 2.1 time: 0.040147 Apr 12 02:00:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 160/489] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:19 2023] GET /compute/v2.1/servers/77f15843-a3a9-4943-a763-0901cd84d7ca => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:00:20 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:00:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-13ed6e54-4177-42c7-ab04-54b9e02780ac req-4c39aa8b-cca4-43f7-a66a-32f1a8523680 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035429 Apr 12 02:00:20 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 164/490] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:20 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:20 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:20 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:20 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] Creating event network-changed:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:00:20 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8169bce2-b845-4fb8-b95b-c258c4b7060e req-a26dace5-2a36-4564-83bb-fdb33a012bde service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.127126 Apr 12 02:00:20 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 167/491] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:20 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:21 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lazy-loading 'services' on Instance uuid 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:00:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b1c446e4-865f-44ab-bb65-8b8b7b22661b tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments/cf91b105-163e-4c1d-a936-6939ccb3db3f" status: 202 len: 0 microversion: 2.1 time: 0.195291 Apr 12 02:00:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 161/492] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:00:21 2023] DELETE /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments/cf91b105-163e-4c1d-a936-6939ccb3db3f => generated 0 bytes in 197 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7479f8d9-ba8f-4789-9b96-0ca2cd8e1eee tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047218 Apr 12 02:00:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 165/493] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:00:21 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:22 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}, {"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:00:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:00:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-733e40e5-68df-4411-807b-bee42a148bc6 req-6dad2869-9b8f-4f33-9ab8-cd949ab2ce0c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.043717 Apr 12 02:00:22 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 168/494] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:22 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:23 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:00:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f7d13973-f5db-4f68-bdee-103e1ece66db req-dbd09026-fecc-452b-a92f-b42c17e73202 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043723 Apr 12 02:00:23 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 162/495] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:23 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:00:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-572d5ce3-16e0-4095-af43-d12aa56b9a0e req-270f484e-c4e0-4d4e-8a43-a5967930e471 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033175 Apr 12 02:00:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 166/496] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4a9d087a-5399-4cfe-b282-d8f0259fd86c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action" status: 202 len: 0 microversion: 2.1 time: 0.169340 Apr 12 02:00:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 169/497] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:00:26 2023] POST /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/action => generated 0 bytes in 171 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:26 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3501da36-fa18-4aec-93fd-06b46908b8d9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1730 microversion: 2.1 time: 0.189481 Apr 12 02:00:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 163/498] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:26 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1730 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:27 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1013259307", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "c12e9557-d89a-46e7-8232-823d5738fa26"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:27 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] validate_networks() for [('c12e9557-d89a-46e7-8232-823d5738fa26', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:00:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-unplugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Creating event network-vif-unplugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:00:27 user devstack@n-api.service[72506]: DEBUG nova.compute.api [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72506) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:00:27 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-80ebc3d2-214e-4cd3-a4b6-6c5c0600a1fe req-2c47946f-639d-476b-9451-6a4ea67e5240 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044377 Apr 12 02:00:27 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 170/499] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:27 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:27 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Getting quotas for project 1dbc688a91f547c0adbf929d0f714d51. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Getting quotas for user cb64ad9101674c1a92d41dfd57ca8e15 and project 1dbc688a91f547c0adbf929d0f714d51. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:00:28 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:00:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f8a7962f-0e2d-4327-bd61-9a31e25c89bd tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 1.104225 Apr 12 02:00:28 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 167/500] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:00:27 2023] POST /compute/v2.1/servers => generated 378 bytes in 1106 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:00:28 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3a82d902-fa0d-4620-84e3-04c56d4cc5b7 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:28 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3a82d902-fa0d-4620-84e3-04c56d4cc5b7 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:28 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:28 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:28 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3a82d902-fa0d-4620-84e3-04c56d4cc5b7 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3a82d902-fa0d-4620-84e3-04c56d4cc5b7 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9" status: 200 len: 1213 microversion: 2.1 time: 0.077294 Apr 12 02:00:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 164/501] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:28 2023] GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 => generated 1213 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:29 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6d8e5ff9-4a8e-4ac6-94ac-0a6eca9b2ff0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 200 len: 1542 microversion: 2.1 time: 0.333647 Apr 12 02:00:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 171/502] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:29 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 1542 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:00:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72507) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:00:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-93bddc13-a4ad-430a-a470-02f30b66b716 req-03c8feb1-4253-4339-b760-4e6ceaed1b4b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041746 Apr 12 02:00:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 168/503] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:29 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] Creating event network-changed:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:00:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-db6759bf-d861-4068-8647-1bb122f1adb6 req-d6c6b01d-2681-4173-a179-9f8ffa0e9d7d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039029 Apr 12 02:00:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 165/504] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:30 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "name": "network-vif-plugged", "status": "completed", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:00:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] Creating event network-vif-plugged:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:00:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-3f94dfd6-27bd-4b83-b75e-2e9fb5b32f66 req-633b6c83-ee2c-45e7-8204-a3fd566b2489 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124579 Apr 12 02:00:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 172/505] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:32 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:33 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:00:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-ee4c0079-f653-45a3-87e2-d79ac208f44a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1431 microversion: 2.1 time: 0.265043 Apr 12 02:00:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 169/506] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:00:33 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1431 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:00:35 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "name": "network-vif-plugged", "status": "completed", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] Creating event network-vif-plugged:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:00:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3e01c401-ea49-4284-a043-4283b9ca34fa req-73833f93-a678-42bb-9631-9c11bda83a34 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041579 Apr 12 02:00:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 166/507] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:35 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:00:39 user devstack@n-api.service[72506]: DEBUG nova.compute.api [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] Instance c712f2db-72cb-4136-a719-56057d82fff6 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72506) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:00:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c516c220-4546-45cc-80cc-0b5e94d3b0fc req-5c69abdd-b889-446c-89f2-2881b7df3a27 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043638 Apr 12 02:00:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 173/508] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:39 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=35,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:00:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:00:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:00:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-2cbffb68-c597-4c78-aa3a-c8cd4a5636c3 req-76bbecf9-8211-4425-b927-7122ff2a2e6d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045491 Apr 12 02:00:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 170/509] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:00:41 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:07 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:07 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:07 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=55,nova_cell1:UPDATE=5,nova_cell1:INSERT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:08 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8e33ede1-6195-4fcf-9b73-d0449c27a2fe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 200 len: 1535 microversion: 2.1 time: 0.210714 Apr 12 02:01:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 174/510] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:07 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 1535 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ec5ebf2d-c876-465b-b1e5-c511502920f1 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1705 microversion: 2.1 time: 0.296056 Apr 12 02:01:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 167/511] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:07 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1705 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-acd11e72-d839-41c9-bf84-9d6f312e371e tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.058240 Apr 12 02:01:11 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 171/512] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:01:11 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:01:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a0458311-43dc-4eb7-95c3-6516a45d9908 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '.version_select at 0x7fb202fd5fc0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a0458311-43dc-4eb7-95c3-6516a45d9908 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "DELETE /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966" status: 204 len: 0 microversion: 2.1 time: 0.194546 Apr 12 02:01:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 175/513] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:14 2023] DELETE /compute/v2.1/images/37146b86-9ec3-440f-95de-7bf8ac2dd966 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:14 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:01:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-946a9f96-d822-47b9-9ae8-dc34206664d6 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 204 len: 0 microversion: 2.1 time: 0.128082 Apr 12 02:01:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 168/514] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:01:14 2023] DELETE /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:14 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e6c9d8100d504d1c8437b627f5564e33', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cdf7e36a5e4b44f690f875999b4b101a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:14 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0c6d976e-f29d-473c-ab33-6d92b22e5069 tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 200 len: 1678 microversion: 2.1 time: 0.185674 Apr 12 02:01:14 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 172/515] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:14 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 1678 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-unplugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] Creating event network-vif-unplugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 02:01:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ea2ff721-7313-47a1-baff-4b19996e7ee7 req-1593a3e6-5636-4cae-8c5b-42b63d09f04f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036674 Apr 12 02:01:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 176/516] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:14 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-deleted", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] Creating event network-vif-deleted:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 02:01:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7393785e-36db-430f-b956-7884e0f6cd2d req-60ffe742-d0c3-44ff-8f9f-5baffbdf9a06 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033113 Apr 12 02:01:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 173/517] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:16 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6222cfdb-8868-44ce-8287-a66068193d24 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.196396 Apr 12 02:01:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 169/518] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:16 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d49cd12-9b4d-4a93-ad5f-bbf0536d5959", "name": "network-vif-plugged", "status": "completed", "tag": "77c37f41-cbd9-4b96-a5e7-103d187fce32"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] Creating event network-vif-plugged:77c37f41-cbd9-4b96-a5e7-103d187fce32 for instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 on user Apr 12 02:01:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-9bf430e6-785d-4ae0-8d2f-c6f083568c65 req-b94bd72c-76ba-494a-8bd7-e54685d577e5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035179 Apr 12 02:01:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 177/519] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:18 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0d2d4700-bfcd-47ca-b46d-bae9cfd7ab89 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9" status: 200 len: 1496 microversion: 2.1 time: 0.172655 Apr 12 02:01:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 174/520] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:18 2023] GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 => generated 1496 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=27,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-50fd7108-69ec-4522-bf91-482295459f34 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.095363 Apr 12 02:01:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 170/521] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:01:29 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-interface => generated 282 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=10 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] Creating event network-changed:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 02:01:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-565041e2-6999-4f32-a1e6-87097cbee095 req-1fd9e6a6-571d-4b41-95e6-87fd3d233400 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036187 Apr 12 02:01:30 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 178/522] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:30 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=21 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-70ff234c-2b31-4328-a8a5-4d0ed66ef6ed tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/ips" status: 200 len: 145 microversion: 2.1 time: 0.044016 Apr 12 02:01:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 175/523] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:01:31 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/ips => generated 145 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7353ae0d-d613-4966-adae-68aa36212f56"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6d97e8af-edb1-43c4-a5e8-8f722d0511b3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.275602 Apr 12 02:01:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 171/524] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:01:33 2023] POST /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments => generated 194 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:33 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6b29a10f-ebb0-450d-b9dd-914d7bb56385 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1431 microversion: 2.1 time: 0.180439 Apr 12 02:01:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 179/525] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:33 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1431 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] Creating event network-changed:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:01:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-91cbec28-1835-420e-af66-cfc503209003 req-d2dfd55a-a951-4751-a46f-e401a42673b3 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033695 Apr 12 02:01:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 176/526] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:35 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lazy-loading 'flavor' on Instance uuid b8c39c58-73b3-4574-9fcf-0c5901a81aa0 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:01:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d3589ec7-2147-4b65-b8e1-a244ffcee1f6 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "POST /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0/action" status: 202 len: 0 microversion: 2.1 time: 0.136192 Apr 12 02:01:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 172/527] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:01:35 2023] POST /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0/action => generated 0 bytes in 138 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d4002087-5105-43a3-8136-d2c17b1661a1 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1437 microversion: 2.1 time: 0.296215 Apr 12 02:01:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 180/528] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:35 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1437 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-unplugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] Creating event network-vif-unplugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:01:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-26157423-e25d-47e2-ae43-b0ab8a294882 req-9d761290-70f1-4a7a-8a81-6959bc3f181c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029412 Apr 12 02:01:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 177/529] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:35 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] Creating event network-changed:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:01:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-794758f5-f282-42dc-8278-2cd72ac65757 req-05bf1f03-b7af-4ffc-8925-4d9548863777 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028357 Apr 12 02:01:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 173/530] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:37 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:01:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a9b4cfaf-e5e8-4d36-b255-0d247cce307f req-4b7e7aff-4eab-4ede-9798-e8194fe3d754 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034146 Apr 12 02:01:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 181/531] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:37 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-47b21d7c-498c-45de-89ac-62961dda7daa tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: .version_select at 0x7fb2030b75b0>, body: {"keypair": {"name": "tempest-keypair-1196686774", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLVpgpX5yJOQZe/kmK27+zCnCpAv8hp5n1Z5z/6JfKL7n0TJHIZd3wBN6t2NstleJwX5uAbZaPwEfvUa6lQVO8DYrciFuHQtC2fskesOJygGb1X0B1D27t35FhlYWtvnlw=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:41 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-47b21d7c-498c-45de-89ac-62961dda7daa tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:01:41 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-47b21d7c-498c-45de-89ac-62961dda7daa tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:01:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-47b21d7c-498c-45de-89ac-62961dda7daa tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.044257 Apr 12 02:01:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 178/532] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:01:41 2023] POST /compute/v2.1/os-keypairs => generated 386 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:42 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1914850466", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "0875dfb2-68e9-4a37-8295-74f038cfba9a"}], "security_groups": [{"name": "tempest-securitygroup--385543299"}], "key_name": "tempest-keypair-1196686774", "user_data": "}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:42 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] validate_networks() for [('0875dfb2-68e9-4a37-8295-74f038cfba9a', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:01:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:01:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-500e18cf-b526-44ba-88f0-b411f2dd8191 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 399 microversion: 2.1 time: 0.585915 Apr 12 02:01:43 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 174/533] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:01:42 2023] POST /compute/v2.1/servers => generated 399 bytes in 587 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:01:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5ce69d28-16aa-48de-b2d1-55927290ca27 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:01:43 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5ce69d28-16aa-48de-b2d1-55927290ca27 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:01:43 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5ce69d28-16aa-48de-b2d1-55927290ca27 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:01:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5ce69d28-16aa-48de-b2d1-55927290ca27 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 200 len: 1236 microversion: 2.1 time: 0.074040 Apr 12 02:01:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 182/534] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:01:43 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 1236 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:01:46 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] Creating event network-changed:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:01:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-915f1c45-79c7-42a2-905d-8a7d7f1ee290 req-b7d394fe-fda1-4676-a900-00cbc5040b67 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033821 Apr 12 02:01:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 179/535] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:46 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "name": "network-vif-plugged", "status": "completed", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] Creating event network-vif-plugged:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:01:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-2e2a7589-aff1-4292-8f90-f246e8269664 req-abcd538b-f477-4315-bbe9-8055a9d351b1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040414 Apr 12 02:01:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 175/536] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:48 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:01:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "name": "network-vif-plugged", "status": "completed", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:01:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=16 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:01:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:01:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:01:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:01:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] Creating event network-vif-plugged:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:01:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-433cd91f-bbeb-4878-a435-70a8de6e114d req-9ac45195-863e-4321-b6b4-b1a537a94fba service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038774 Apr 12 02:01:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 183/537] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:01:50 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=19,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] [instance: 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] HTTP exception thrown: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Request method failure captured: Apr 12 02:02:04 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 HTTP/1.1 Apr 12 02:02:04 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:02:04 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:02:04 user devstack@n-api.service[72507]: Connection: close Apr 12 02:02:04 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:02:04 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:02:04 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:02:04 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:02:04 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg87si8EAwzO9B5_bdnjhuOYhBq45cvWmCbmunFHim9tyfWYFqnOi_w5HFNKaSHszOZcINwivyWnFmmnGl_r3yKBdZQ9rWBaJMMuX4Mo7b3vWyd6hwfrnx3HiK4juBhfQiAxA7JDw5Wr7-1ttxXAPaGy_3qTxn5RKknxbEjr6l5nI6I Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Project-Id: cdf7e36a5e4b44f690f875999b4b101a Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerStableDeviceRescueTest-1741311402 Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/cdf7e36a5e4b44f690f875999b4b101a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/cdf7e36a5e4b44f690f875999b4b101a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/cdf7e36a5e4b44f690f875999b4b101a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerStableDeviceRescueTest-1741311402 Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Tenant-Id: cdf7e36a5e4b44f690f875999b4b101a Apr 12 02:02:04 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerStableDeviceRescueTest-1741311402 Apr 12 02:02:04 user devstack@n-api.service[72507]: X-User: tempest-ServerStableDeviceRescueTest-1741311402-project-member Apr 12 02:02:04 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:02:04 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:02:04 user devstack@n-api.service[72507]: X-User-Id: e6c9d8100d504d1c8437b627f5564e33 Apr 12 02:02:04 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerStableDeviceRescueTest-1741311402-project-member Apr 12 02:02:04 user devstack@n-api.service[72507]: method: > Apr 12 02:02:04 user devstack@n-api.service[72507]: exception: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. Apr 12 02:02:04 user devstack@n-api.service[72507]: action_args: {'id': '0d49cd12-9b4d-4a93-ad5f-bbf0536d5959'} Apr 12 02:02:04 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. Apr 12 02:02:04 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:04 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Returning 404 to user: Instance 0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:02:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b1b325c6-d833-4e5c-9391-c635b8f16bcd tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959" status: 404 len: 111 microversion: 2.1 time: 0.035845 Apr 12 02:02:04 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 180/538] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:04 2023] GET /compute/v2.1/servers/0d49cd12-9b4d-4a93-ad5f-bbf0536d5959 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:02:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=25,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=36,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:04 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-97cca893-7c04-40c2-acad-57119251cc0a tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] Calling method '.version_select at 0x7fb203042200>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:04 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-97cca893-7c04-40c2-acad-57119251cc0a tempest-ServerStableDeviceRescueTest-1741311402 tempest-ServerStableDeviceRescueTest-1741311402-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-682814124" status: 202 len: 0 microversion: 2.1 time: 0.037500 Apr 12 02:02:04 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 176/539] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:02:04 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-682814124 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=14 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cd375f6e-5135-458c-83c0-fdf2e6e3f8bb tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.108399 Apr 12 02:02:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 184/540] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:02:08 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-interface => generated 283 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] HTTP exception thrown: Cannot 'detach_volume' instance 84b4902c-24d9-4e96-b51f-e93e913aa681 while it is in vm_state rescued Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Request method failure captured: Apr 12 02:02:08 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments/f6f462d6-7d7c-4843-97c3-d0ded790997f HTTP/1.1 Apr 12 02:02:08 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:02:08 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:02:08 user devstack@n-api.service[72507]: Connection: close Apr 12 02:02:08 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:02:08 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:02:08 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:02:08 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:02:08 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg44PjYOKvn626k4LU-GdnrYvKUw0JPm986bERmaP8HWBpoYllBF-zT9F33eDRWVPNLXGnPnqSXLqxvxPxbpnKT-DsqjWHFcOAkLCgQVhgVNvj2O847gDgRAlXz1fgsLlBnnV14mxHU4brxBUpZaN6c1uNs-kqCIbkSyeXxWQGBSWz0 Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Project-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Tenant-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:02:08 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:02:08 user devstack@n-api.service[72507]: X-User: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:02:08 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:02:08 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:02:08 user devstack@n-api.service[72507]: X-User-Id: 889189da060e4b54a48ce9a013e04a3d Apr 12 02:02:08 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:02:08 user devstack@n-api.service[72507]: method: > Apr 12 02:02:08 user devstack@n-api.service[72507]: exception: Cannot 'detach_volume' instance 84b4902c-24d9-4e96-b51f-e93e913aa681 while it is in vm_state rescued Apr 12 02:02:08 user devstack@n-api.service[72507]: action_args: {'server_id': '84b4902c-24d9-4e96-b51f-e93e913aa681', 'id': 'f6f462d6-7d7c-4843-97c3-d0ded790997f'} Apr 12 02:02:08 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 in vm_state rescued. Cannot detach_volume while the instance is in this state. Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 84b4902c-24d9-4e96-b51f-e93e913aa681 while it is in vm_state rescued Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 84b4902c-24d9-4e96-b51f-e93e913aa681 while it is in vm_state rescued Apr 12 02:02:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:02:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Returning 409 to user: Cannot 'detach_volume' instance 84b4902c-24d9-4e96-b51f-e93e913aa681 while it is in vm_state rescued {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:02:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-483096ef-377c-47cd-b94c-103ebfdc8cee tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments/f6f462d6-7d7c-4843-97c3-d0ded790997f" status: 409 len: 152 microversion: 2.1 time: 0.116734 Apr 12 02:02:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 181/541] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:02:08 2023] DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments/f6f462d6-7d7c-4843-97c3-d0ded790997f => generated 152 bytes in 118 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-59294830-9287-4136-8831-7ed4a9a64297 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/action" status: 202 len: 0 microversion: 2.1 time: 0.088406 Apr 12 02:02:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 177/542] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:02:08 2023] POST /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f08dda12-a726-479d-837d-b611d488bed4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1713 microversion: 2.1 time: 0.319348 Apr 12 02:02:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 185/543] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:08 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1713 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] Creating event network-changed:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-9b9b939a-8d57-47a0-84dd-ee9dae454b39 req-f10572dc-dfac-4ac8-b3ee-a386c4747c8a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037581 Apr 12 02:02:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 182/544] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:09 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a64508a5-e36b-4883-b69b-ca9e27941be7 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/ips" status: 200 len: 105 microversion: 2.1 time: 0.060773 Apr 12 02:02:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 178/545] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:02:09 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/ips => generated 105 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-unplugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] Creating event network-vif-unplugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:02:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-2ed8c89c-d571-4a54-89e6-c7f94b8843bb req-4f756b7a-1818-4aea-bb04-cc24a9cc89d0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032234 Apr 12 02:02:09 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 186/546] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:09 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6f0ad3fc-378c-4429-b7b6-f5228cf3a42d"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-17b3e4f0-bdb7-4f7c-a41f-c567da47c359 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.287064 Apr 12 02:02:11 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 183/547] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:02:11 2023] POST /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 194 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}, {"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:02:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:02:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c7505392-2c65-4f2c-94e9-2e92dc5f9ae9 req-62e5c034-3c54-440b-8594-6fd3a4e9dce0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.039196 Apr 12 02:02:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 179/548] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:11 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:12 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-34100019-1198-4b96-bf2c-abd6140b3bb4 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb202fd4790>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:12 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-34100019-1198-4b96-bf2c-abd6140b3bb4 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/27a4c401-6715-4662-9bb6-02f2e14c1a10" status: 200 len: 350 microversion: 2.1 time: 0.057798 Apr 12 02:02:12 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 187/549] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:02:12 2023] GET /compute/v2.1/os-volumes/27a4c401-6715-4662-9bb6-02f2e14c1a10 => generated 350 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-97dc6147-c5ab-46c7-bc16-4314af9b4335 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb20349ee60>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-97dc6147-c5ab-46c7-bc16-4314af9b4335 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/aa6c4f91-3821-4f4b-95e6-2bb5bcc9b09b" status: 200 len: 350 microversion: 2.1 time: 0.058769 Apr 12 02:02:13 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 184/550] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:02:13 2023] GET /compute/v2.1/os-volumes/aa6c4f91-3821-4f4b-95e6-2bb5bcc9b09b => generated 350 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:02:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-bcc1484b-a2e6-4c50-ad6d-cd03198fb042 req-de45d38d-08cc-4d0f-aafb-81cb49b4ee68 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034115 Apr 12 02:02:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 180/551] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2de1af87-5885-41e3-a473-f3fffec9fb27 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb2032d2320>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-2de1af87-5885-41e3-a473-f3fffec9fb27 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/ca2dee88-3f1d-481f-88a7-0458bf7e1896" status: 200 len: 351 microversion: 2.1 time: 0.056690 Apr 12 02:02:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 188/552] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:02:14 2023] GET /compute/v2.1/os-volumes/ca2dee88-3f1d-481f-88a7-0458bf7e1896 => generated 351 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-03105c18-9dfb-40ab-b203-a03a7e3a813f tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb2034079a0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-03105c18-9dfb-40ab-b203-a03a7e3a813f tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.113751 Apr 12 02:02:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 185/553] 10.0.0.210 () {62 vars in 1211 bytes} [Wed Apr 12 02:02:14 2023] GET /compute/v2.1/os-volumes => generated 1034 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e200296d-2f22-4a3f-a469-539afc536c1e tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb203043400>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e200296d-2f22-4a3f-a469-539afc536c1e tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.109486 Apr 12 02:02:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 181/554] 10.0.0.210 () {62 vars in 1226 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f472b39a-97b6-4874-bca7-8a01a200a482 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb203151ea0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f472b39a-97b6-4874-bca7-8a01a200a482 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.084121 Apr 12 02:02:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 189/555] 10.0.0.210 () {62 vars in 1211 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes => generated 1034 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2a3092b4-5037-44e7-85d9-12fc5bb93ef7 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb2030b79a0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2a3092b4-5037-44e7-85d9-12fc5bb93ef7 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.095963 Apr 12 02:02:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 186/556] 10.0.0.210 () {62 vars in 1244 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0cc18434-ac21-43f2-86b8-38c07e005fea tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb203091a20>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0cc18434-ac21-43f2-86b8-38c07e005fea tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.087929 Apr 12 02:02:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 182/557] 10.0.0.210 () {62 vars in 1247 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-33afbbff-9028-41ff-a986-21db7c3becca tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb202fd5900>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-33afbbff-9028-41ff-a986-21db7c3becca tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.095350 Apr 12 02:02:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 190/558] 10.0.0.210 () {62 vars in 1232 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3df737e8-34c6-4bd3-b032-81162e1f916f tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb202f09bd0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3df737e8-34c6-4bd3-b032-81162e1f916f tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.102021 Apr 12 02:02:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 187/559] 10.0.0.210 () {62 vars in 1265 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-75b51ad3-3942-42b0-bbfa-9dc667e694c7 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] Calling method '.version_select at 0x7fb20311ef80>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-75b51ad3-3942-42b0-bbfa-9dc667e694c7 tempest-VolumesTestJSON-72018261 tempest-VolumesTestJSON-72018261-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.118639 Apr 12 02:02:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 183/560] 10.0.0.210 () {62 vars in 1232 bytes} [Wed Apr 12 02:02:15 2023] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:17 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e8936902-b05b-471d-aff3-1be8ba1a2080 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.162232 Apr 12 02:02:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 191/561] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:16 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:17 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5831426e-3585-4831-939a-6460f9481747 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1723 microversion: 2.1 time: 0.257050 Apr 12 02:02:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 188/562] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:17 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1723 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "df839fb4-cc91-451c-97ba-274b05b51147", "device": "/dev/vdb"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1f4b7729-f0c0-4bb1-8a50-05b77d00b8d8 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "POST /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.487847 Apr 12 02:02:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 184/563] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:02:18 2023] POST /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9/os-volume_attachments => generated 194 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-12ccaeee-3595-4db5-acef-d0bfa0b70743 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046951 Apr 12 02:02:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 192/564] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:02:19 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f8c6e339-39d2-4ade-bb65-143ea32dadfe tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d" status: 200 len: 194 microversion: 2.1 time: 0.048305 Apr 12 02:02:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 189/565] 10.0.0.210 () {62 vars in 1490 bytes} [Wed Apr 12 02:02:19 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "86b90a95-c0be-4069-b9ee-7f0edf77dee7"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ad8e127c-78b0-43a7-978d-c61c5f6ffb37 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "POST /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.481291 Apr 12 02:02:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 185/566] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:02:20 2023] POST /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 194 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:25 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a010bfa2-c044-41a6-8320-732739c54595 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1497 microversion: 2.1 time: 0.223311 Apr 12 02:02:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 193/567] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:25 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1497 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-667db76f-3f28-4d1e-9fd1-0fbb232594bf tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.048548 Apr 12 02:02:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 190/568] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:02:29 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 371 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4f960cef-3f67-40ab-9632-69a281e4f157 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d" status: 200 len: 194 microversion: 2.1 time: 0.055536 Apr 12 02:02:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 186/569] 10.0.0.210 () {62 vars in 1490 bytes} [Wed Apr 12 02:02:29 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d => generated 194 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:29 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lazy-loading 'services' on Instance uuid 8c37378d-17a9-42a1-abfc-00037e305d34 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:02:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f8e65042-86cd-4b47-ba2a-120e104b1143 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d" status: 202 len: 0 microversion: 2.1 time: 0.182457 Apr 12 02:02:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 194/570] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:02:29 2023] DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/6f0ad3fc-378c-4429-b7b6-f5228cf3a42d => generated 0 bytes in 184 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d12e0e9c-ab0d-4cc2-a202-5b3c3e9fc620 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/86b90a95-c0be-4069-b9ee-7f0edf77dee7" status: 200 len: 194 microversion: 2.1 time: 0.058529 Apr 12 02:02:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 191/571] 10.0.0.210 () {62 vars in 1490 bytes} [Wed Apr 12 02:02:31 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/86b90a95-c0be-4069-b9ee-7f0edf77dee7 => generated 194 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:31 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lazy-loading 'services' on Instance uuid 8c37378d-17a9-42a1-abfc-00037e305d34 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:02:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e2d037ca-7619-4bc7-8ed8-856e038f4095 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/86b90a95-c0be-4069-b9ee-7f0edf77dee7" status: 202 len: 0 microversion: 2.1 time: 0.166906 Apr 12 02:02:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 187/572] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:02:31 2023] DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments/86b90a95-c0be-4069-b9ee-7f0edf77dee7 => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:33 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-55bf64bd-0fcd-4712-8c3b-6c927d6519db tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 200 len: 1545 microversion: 2.1 time: 0.154495 Apr 12 02:02:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 195/573] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:33 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 1545 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5f3d069b-ac03-4b64-93d7-de2b8a7bb6ad tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053302 Apr 12 02:02:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 192/574] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:02:34 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b46489ef-25f0-4120-9f79-5d98e6939d35 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046039 Apr 12 02:02:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 188/575] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:02:34 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:02:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cde5949a-c2b3-4985-b789-73e897e1dd10 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 204 len: 0 microversion: 2.1 time: 0.119557 Apr 12 02:02:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 196/576] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:02:34 2023] DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lazy-loading 'services' on Instance uuid cf8cad76-0623-48a0-8b77-bb94218e17a9 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd2ca4855c3e9461cb27c88469086d7d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c34bd636a2a54b249dca9ce9027ebf4b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-cccec180-6974-4ff3-8e40-faf8c23c383c tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 200 len: 1659 microversion: 2.1 time: 0.257448 Apr 12 02:02:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 193/577] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:34 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 1659 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-unplugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2c3c71e4-b6f4-488c-a528-ea58de7dc62e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9/os-volume_attachments/df839fb4-cc91-451c-97ba-274b05b51147" status: 202 len: 0 microversion: 2.1 time: 0.191894 Apr 12 02:02:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 189/578] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:02:34 2023] DELETE /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9/os-volume_attachments/df839fb4-cc91-451c-97ba-274b05b51147 => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] Creating event network-vif-unplugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-2b168b96-9939-4be7-a1f3-b626edca795f req-512fcbeb-7b86-4bd1-ac24-5d641ca4dbb4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030853 Apr 12 02:02:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 197/579] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:34 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=65,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=23 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-77fe7136-9e6c-4a81-8703-f03daf8d871e tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9" status: 204 len: 0 microversion: 2.1 time: 0.140710 Apr 12 02:02:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 194/580] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:02:34 2023] DELETE /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=5,nova_cell1:INSERT=5 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:35 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cb64ad9101674c1a92d41dfd57ca8e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dbc688a91f547c0adbf929d0f714d51', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a9d465ff-b69b-4f21-bbe2-45d2d1966dc2 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9" status: 200 len: 1548 microversion: 2.1 time: 0.178376 Apr 12 02:02:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 190/581] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:34 2023] GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 => generated 1548 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:02:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-deleted", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] Creating event network-vif-deleted:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-61d45770-ef08-46e8-a556-f483782d1df8 req-99206ff1-fec8-4231-a1d6-995125afddf7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034220 Apr 12 02:02:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 198/582] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:36 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}, {"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}, {"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}, {"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-unplugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}, {"server_uuid": "8c37378d-17a9-42a1-abfc-00037e305d34", "name": "network-vif-plugged", "status": "completed", "tag": "482853e3-5a37-4420-a614-c01ba39cfb74"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Creating event network-vif-unplugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] Creating event network-vif-plugged:482853e3-5a37-4420-a614-c01ba39cfb74 for instance 8c37378d-17a9-42a1-abfc-00037e305d34 on user Apr 12 02:02:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b6b869d4-9c6c-4f06-b091-03b595a47536 req-bed1d834-734d-44d9-9411-dfce358de5a4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 869 microversion: 2.1 time: 0.038529 Apr 12 02:02:36 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 195/583] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:36 2023] POST /compute/v2.1/os-server-external-events => generated 869 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "name": "network-vif-unplugged", "status": "completed", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] Creating event network-vif-unplugged:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:02:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-fc6d09ae-200c-4e42-bd43-a1b1b953457e req-825ab640-d1ea-4c7f-9e66-ba32a97fb8fa service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037581 Apr 12 02:02:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 191/584] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:37 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "name": "network-vif-deleted", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=28,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=65,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] Creating event network-vif-deleted:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:02:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-1071aedc-f998-4694-bd81-82f1316a1648 req-dd4b6c86-18b3-4fc3-854f-4fcac3d63898 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043612 Apr 12 02:02:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 199/585] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:38 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf8cad76-0623-48a0-8b77-bb94218e17a9", "name": "network-vif-plugged", "status": "completed", "tag": "409d9b69-5424-4507-8052-591629d7c44c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:02:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] Creating event network-vif-plugged:409d9b69-5424-4507-8052-591629d7c44c for instance cf8cad76-0623-48a0-8b77-bb94218e17a9 on user Apr 12 02:02:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-907077ae-8fcd-4ceb-929d-dc760363105a req-165e8030-0f7a-44a8-bd60-026ef81ed43d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033167 Apr 12 02:02:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 196/586] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:02:39 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=7 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:02:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:02:59 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:02:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-26bc1848-8523-4ad0-bb9a-b294cd8a0d7a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1719 microversion: 2.1 time: 0.409485 Apr 12 02:02:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 192/587] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:02:58 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1719 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e267d0c0-f840-426e-beb4-51da19ab0bb9 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: .version_select at 0x7fb2030897e0>, body: {"keypair": {"name": "tempest-SnapshotDataIntegrityTests-522022221", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBzPcQjTv2Jael0y5wrvN4uSHK22wC+4MQYihJ4lkSsarIJtgTRNzEUL9q5zwomDyXY8RX+h7kFLUBsKlQRa2wYp3AwqYZTBzQkYcWYnC7AOOfvjs6Z8ppYHAJYpNSM3yg=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:05 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-e267d0c0-f840-426e-beb4-51da19ab0bb9 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Getting quotas for project 910948b97a01432fb70094eb6ec30379. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:03:05 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-e267d0c0-f840-426e-beb4-51da19ab0bb9 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Getting quotas for user 34ca83258ae04dee81b8552a6c7e3c2f and project 910948b97a01432fb70094eb6ec30379. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:03:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e267d0c0-f840-426e-beb4-51da19ab0bb9 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.250007 Apr 12 02:03:05 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 200/588] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:03:05 2023] POST /compute/v2.1/os-keypairs => generated 404 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SnapshotDataIntegrityTests-server-1689225161", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "key_name": "tempest-SnapshotDataIntegrityTests-522022221", "security_groups": [{"name": "tempest-secgroup-smoke-1660787941"}], "networks": [{"uuid": "6f35d61b-46c4-4830-984f-93b7bfa8836c"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:07 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] validate_networks() for [('6f35d61b-46c4-4830-984f-93b7bfa8836c', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Getting quotas for project 910948b97a01432fb70094eb6ec30379. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Getting quotas for user 34ca83258ae04dee81b8552a6c7e3c2f and project 910948b97a01432fb70094eb6ec30379. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:03:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:03:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-848e1b5e-ea85-475f-a095-d7d75c0e5a50 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.855859 Apr 12 02:03:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 197/589] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:03:07 2023] POST /compute/v2.1/servers => generated 404 bytes in 858 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Apr 12 02:03:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e79d9ee5-a090-4139-be68-1ee6ed430808 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e79d9ee5-a090-4139-be68-1ee6ed430808 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:08 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-e79d9ee5-a090-4139-be68-1ee6ed430808 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '34ca83258ae04dee81b8552a6c7e3c2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '910948b97a01432fb70094eb6ec30379', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:03:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e79d9ee5-a090-4139-be68-1ee6ed430808 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 200 len: 1256 microversion: 2.1 time: 0.071906 Apr 12 02:03:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 193/590] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:08 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 1256 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] Creating event network-changed:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:03:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-01ace1f9-7b83-48dd-983c-4b5f73b8adcb req-6f0f510b-87fb-47bd-aba7-d864b54e285c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030983 Apr 12 02:03:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 201/591] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:10 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "name": "network-vif-plugged", "status": "completed", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] Creating event network-vif-plugged:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:03:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-1ffda399-04c3-4d60-9b74-b0215feca3c8 req-08047fb1-212a-4df1-bb5a-8898a3559bc5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030720 Apr 12 02:03:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 198/592] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:12 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "name": "network-vif-plugged", "status": "completed", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:15 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] Creating event network-vif-plugged:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:03:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-61d447f3-b5a6-4fc5-8581-f0d7b8c117e3 req-caa2148e-6e46-4f2e-98f9-c44375b2a88a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034715 Apr 12 02:03:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 194/593] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:03:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a848fdf1-9f84-43e0-b855-9188e5e345c5 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 204 len: 0 microversion: 2.1 time: 0.117886 Apr 12 02:03:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 202/594] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:03:17 2023] DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:17 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:03:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d3a21c64-ae58-42ea-91af-6d68dafced21 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 200 len: 1729 microversion: 2.1 time: 0.210213 Apr 12 02:03:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 199/595] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:17 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 1729 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-unplugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] Creating event network-vif-unplugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:03:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-bb66757d-9e22-42de-a3c4-6b53213aa243 req-a8ea294e-1e8d-4d0f-8ebf-07fe45d6f215 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039032 Apr 12 02:03:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 195/596] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:17 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-deleted", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] Creating event network-vif-deleted:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:03:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-19da49c7-71ba-4cfe-8520-65fe837cbf44 req-fadcb432-534f-4cec-996c-4e67b5ebf0f2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.027391 Apr 12 02:03:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 203/597] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:19 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c712f2db-72cb-4136-a719-56057d82fff6", "name": "network-vif-plugged", "status": "completed", "tag": "365b7a91-793f-4aaf-af62-4e1c45d300e8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] Creating event network-vif-plugged:365b7a91-793f-4aaf-af62-4e1c45d300e8 for instance c712f2db-72cb-4136-a719-56057d82fff6 on user Apr 12 02:03:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-5d437e3f-36c1-4379-b19c-68b0fb285ca2 req-4705c6cc-75c5-4c0e-bef7-829284f3b850 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029792 Apr 12 02:03:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 200/598] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:19 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:24 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:03:24 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 HTTP/1.1 Apr 12 02:03:24 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:03:24 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:03:24 user devstack@n-api.service[72505]: Connection: close Apr 12 02:03:24 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:03:24 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:03:24 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:03:24 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:03:24 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:24 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:24 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:24 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:03:24 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:03:24 user devstack@n-api.service[72505]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:03:24 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:24 user devstack@n-api.service[72505]: method: > Apr 12 02:03:24 user devstack@n-api.service[72505]: exception: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:24 user devstack@n-api.service[72505]: action_args: {'id': '8c37378d-17a9-42a1-abfc-00037e305d34'} Apr 12 02:03:24 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:24 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:24 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-28f72d16-7d24-450f-842e-2b1c6e6631ea tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 404 len: 111 microversion: 2.1 time: 0.033381 Apr 12 02:03:24 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 196/599] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:24 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5f373c29-e2fa-4952-88a9-5b0490c142d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '.version_select at 0x7fb202fd7d00>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5f373c29-e2fa-4952-88a9-5b0490c142d4 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-155530966" status: 202 len: 0 microversion: 2.1 time: 0.024202 Apr 12 02:03:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 204/600] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:03:24 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-155530966 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] [instance: cf8cad76-0623-48a0-8b77-bb94218e17a9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] HTTP exception thrown: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Request method failure captured: Apr 12 02:03:25 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 HTTP/1.1 Apr 12 02:03:25 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:03:25 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:03:25 user devstack@n-api.service[72507]: Connection: close Apr 12 02:03:25 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:03:25 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:03:25 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:03:25 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:03:25 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg_7Gp7NjeUBhd0X0pUaopW-UaAw4pgOJoXNZ04FgC8Oo6fg8K_1Yd-2SM7IC8fO0oO74CEV4gVk_wJsvjCqLLZWW-VoyWbSBWFNLjgAG7qt1tQOw7eObsk3ZNI56cbByTskLXt0OJGRFO4NH2xshxvfG7RoCISud0gKS6NTdr-lwFE Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Project-Id: 1dbc688a91f547c0adbf929d0f714d51 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/1dbc688a91f547c0adbf929d0f714d51"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Tenant-Id: 1dbc688a91f547c0adbf929d0f714d51 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1019440197 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-User: tempest-VolumesSnapshotTestJSON-1019440197-project-member Apr 12 02:03:25 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:03:25 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:03:25 user devstack@n-api.service[72507]: X-User-Id: cb64ad9101674c1a92d41dfd57ca8e15 Apr 12 02:03:25 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesSnapshotTestJSON-1019440197-project-member Apr 12 02:03:25 user devstack@n-api.service[72507]: method: > Apr 12 02:03:25 user devstack@n-api.service[72507]: exception: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. Apr 12 02:03:25 user devstack@n-api.service[72507]: action_args: {'id': 'cf8cad76-0623-48a0-8b77-bb94218e17a9'} Apr 12 02:03:25 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. Apr 12 02:03:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] Returning 404 to user: Instance cf8cad76-0623-48a0-8b77-bb94218e17a9 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-acb66762-a080-41a4-99a7-6f558caea234 tempest-VolumesSnapshotTestJSON-1019440197 tempest-VolumesSnapshotTestJSON-1019440197-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9" status: 404 len: 111 microversion: 2.1 time: 0.035867 Apr 12 02:03:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 201/601] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:25 2023] GET /compute/v2.1/servers/cf8cad76-0623-48a0-8b77-bb94218e17a9 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=72505) unshelve /opt/stack/nova/nova/compute/api.py:4596}} Apr 12 02:03:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-89b36eeb-1814-42c7-8d4a-b2fb4d92dd11 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "POST /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0/action" status: 202 len: 0 microversion: 2.1 time: 0.190652 Apr 12 02:03:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 197/602] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:03:26 2023] POST /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0/action => generated 0 bytes in 193 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:26 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:03:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-fbecc6e8-2f96-4b0b-af81-ad9bdd6d6425 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1505 microversion: 2.1 time: 0.172496 Apr 12 02:03:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 205/603] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:26 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1505 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:30 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:03:30 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 HTTP/1.1 Apr 12 02:03:30 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:03:30 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:03:30 user devstack@n-api.service[72507]: Connection: close Apr 12 02:03:30 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:03:30 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:03:30 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:03:30 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:03:30 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:30 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:30 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:03:30 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:03:30 user devstack@n-api.service[72507]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:03:30 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:30 user devstack@n-api.service[72507]: method: > Apr 12 02:03:30 user devstack@n-api.service[72507]: exception: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72507]: action_args: {'id': '8c37378d-17a9-42a1-abfc-00037e305d34'} Apr 12 02:03:30 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:30 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-5573e141-8b7c-455e-90ae-1a66fcc06978 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 404 len: 111 microversion: 2.1 time: 0.027660 Apr 12 02:03:30 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 202/604] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:03:30 2023] DELETE /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: 8c37378d-17a9-42a1-abfc-00037e305d34] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:03:30 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 HTTP/1.1 Apr 12 02:03:30 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:03:30 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:03:30 user devstack@n-api.service[72505]: Connection: close Apr 12 02:03:30 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:03:30 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:03:30 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:03:30 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:03:30 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:30 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:30 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:30 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:03:30 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:03:30 user devstack@n-api.service[72505]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:03:30 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:30 user devstack@n-api.service[72505]: method: > Apr 12 02:03:30 user devstack@n-api.service[72505]: exception: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72505]: action_args: {'id': '8c37378d-17a9-42a1-abfc-00037e305d34'} Apr 12 02:03:30 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. Apr 12 02:03:30 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:03:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance 8c37378d-17a9-42a1-abfc-00037e305d34 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-91d2fc16-d5bd-47aa-95cd-7fbba74d2506 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34" status: 404 len: 111 microversion: 2.1 time: 0.031226 Apr 12 02:03:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 198/605] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:30 2023] GET /compute/v2.1/servers/8c37378d-17a9-42a1-abfc-00037e305d34 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:03:32 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b HTTP/1.1 Apr 12 02:03:32 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:03:32 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:03:32 user devstack@n-api.service[72506]: Connection: close Apr 12 02:03:32 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:03:32 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:03:32 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:03:32 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:03:32 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:32 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:32 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:03:32 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:03:32 user devstack@n-api.service[72506]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:03:32 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:32 user devstack@n-api.service[72506]: method: > Apr 12 02:03:32 user devstack@n-api.service[72506]: exception: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72506]: action_args: {'id': 'cb89325c-680a-4413-8e94-4eddfaa79c6b'} Apr 12 02:03:32 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-2f178a38-9d3d-424c-8a4a-34b6e336b61f tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 404 len: 111 microversion: 2.1 time: 0.031107 Apr 12 02:03:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 206/606] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:03:32 2023] DELETE /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] [instance: cb89325c-680a-4413-8e94-4eddfaa79c6b] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] HTTP exception thrown: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Request method failure captured: Apr 12 02:03:32 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b HTTP/1.1 Apr 12 02:03:32 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:03:32 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:03:32 user devstack@n-api.service[72507]: Connection: close Apr 12 02:03:32 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:03:32 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:03:32 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:03:32 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:03:32 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg46SxgkkSmiCZuHDku-EzUJDzuj5lmuw5BQPk9mWBFbQqG6ickCrN2rps-CAbD70LtbXl5B0bdTNaeFkjOqPkU82hLbpfkRhEIKSB1u2P7i5TckyrwPwWXOm7UdHV0capkiuHPHIp96YLHV8vrcagMFLZmjS1Dh5Qauh-yA2jZc39M Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Project-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/c34bd636a2a54b249dca9ce9027ebf4b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Tenant-Id: c34bd636a2a54b249dca9ce9027ebf4b Apr 12 02:03:32 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeTestJSON-913560355 Apr 12 02:03:32 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:32 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:03:32 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:03:32 user devstack@n-api.service[72507]: X-User-Id: d2ca4855c3e9461cb27c88469086d7d8 Apr 12 02:03:32 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeTestJSON-913560355-project-member Apr 12 02:03:32 user devstack@n-api.service[72507]: method: > Apr 12 02:03:32 user devstack@n-api.service[72507]: exception: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72507]: action_args: {'id': 'cb89325c-680a-4413-8e94-4eddfaa79c6b'} Apr 12 02:03:32 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. Apr 12 02:03:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:03:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] Returning 404 to user: Instance cb89325c-680a-4413-8e94-4eddfaa79c6b could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-121af8cc-8329-4e8d-baaf-b0e1c1705ad0 tempest-AttachVolumeTestJSON-913560355 tempest-AttachVolumeTestJSON-913560355-project-member] 10.0.0.210 "GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b" status: 404 len: 111 microversion: 2.1 time: 0.027169 Apr 12 02:03:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 203/607] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:32 2023] GET /compute/v2.1/servers/cb89325c-680a-4413-8e94-4eddfaa79c6b => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:03:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] Creating event network-changed:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:03:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f4af25a6-f324-44c5-a831-568f8e080510 req-c11d30f6-0230-4160-a1dc-3ad3f7c74991 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053411 Apr 12 02:03:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 199/608] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:33 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-537fc7ca-0560-44d8-95e5-121f9b94910e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.117603 Apr 12 02:03:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 207/609] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:03:33 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/os-interface => generated 283 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] Creating event network-changed:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:03:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e1f8f4c6-d72f-42b3-9176-8d1eee0a086b req-27af9730-0301-4ec5-b929-bbb3da1d56f2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028301 Apr 12 02:03:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 204/610] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:34 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9ad17ce5-eae3-4800-a968-f2bfdd71f305 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/ips" status: 200 len: 108 microversion: 2.1 time: 0.040708 Apr 12 02:03:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 200/611] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:03:34 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/ips => generated 108 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7353ae0d-d613-4966-adae-68aa36212f56"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:03:35 user devstack@n-api.service[72506]: request: POST /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/os-volume_attachments HTTP/1.1 Apr 12 02:03:35 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:03:35 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:03:35 user devstack@n-api.service[72506]: Connection: close Apr 12 02:03:35 user devstack@n-api.service[72506]: Content-Length: 74 Apr 12 02:03:35 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:03:35 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:03:35 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:03:35 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:03:35 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:03:35 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:03:35 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:03:35 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:03:35 user devstack@n-api.service[72506]: Apr 12 02:03:35 user devstack@n-api.service[72506]: {"volumeAttachment": {"volumeId": "7353ae0d-d613-4966-adae-68aa36212f56"}} Apr 12 02:03:35 user devstack@n-api.service[72506]: method: > Apr 12 02:03:35 user devstack@n-api.service[72506]: exception: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd Apr 12 02:03:35 user devstack@n-api.service[72506]: action_args: {'server_id': '0fa3c7e6-a1a9-490e-ae02-8b1359736485', 'body': {'volumeAttachment': {'volumeId': '7353ae0d-d613-4966-adae-68aa36212f56'}}} Apr 12 02:03:35 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5139, in attach_volume Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4979, in _check_volume_already_attached Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd Apr 12 02:03:35 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 400 to user: Invalid volume: volume 7353ae0d-d613-4966-adae-68aa36212f56 is already attached to instances: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:03:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a756cc87-195e-4158-8d71-6d5e28785d8e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.104139 Apr 12 02:03:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 208/612] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:03:34 2023] POST /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485/os-volume_attachments => generated 174 bytes in 106 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:03:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0ad3fb24-1598-4504-8931-4549f1d318e1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 204 len: 0 microversion: 2.1 time: 0.137718 Apr 12 02:03:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 205/613] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:03:35 2023] DELETE /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:03:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-aa5b2abf-ca32-44ad-a7ed-d7054478925b tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 200 len: 1551 microversion: 2.1 time: 0.241852 Apr 12 02:03:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 201/614] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:35 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 1551 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "name": "network-vif-unplugged", "status": "completed", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] Creating event network-vif-unplugged:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:03:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ab27b787-0db3-4ff7-b4e2-af32197d080f req-ab6828dd-0e9f-4507-ad69-96d857072f7b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031319 Apr 12 02:03:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 209/615] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:35 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:37 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:03:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-2e0de537-d425-49b6-9afa-31db03ef7552 req-ee03ae3f-86f8-4076-b5d9-a2a938e182ba service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040605 Apr 12 02:03:37 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 206/616] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:37 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "name": "network-vif-plugged", "status": "completed", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}, {"server_uuid": "0fa3c7e6-a1a9-490e-ae02-8b1359736485", "name": "network-vif-deleted", "tag": "bee74d19-41d7-4f0a-8ead-0f835edb08f5"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Creating event network-vif-plugged:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:03:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] Creating event network-vif-deleted:bee74d19-41d7-4f0a-8ead-0f835edb08f5 for instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 on user Apr 12 02:03:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8b1b1a03-3236-4dc1-9da0-7bc159eaa311 req-88e25041-3755-4297-ab3b-8dca5bb1adf9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.039126 Apr 12 02:03:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 202/617] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:37 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:39 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=41,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:03:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-fb5e1a1f-6624-49f6-a7c5-f211d1e18936 req-44de4984-0f47-44eb-a353-fd4bb9cb5022 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033152 Apr 12 02:03:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 210/618] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:03:39 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:58 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '34ca83258ae04dee81b8552a6c7e3c2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '910948b97a01432fb70094eb6ec30379', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:03:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6e604a89-940f-46b9-a107-e5cbfa449ae7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 200 len: 1568 microversion: 2.1 time: 0.193208 Apr 12 02:03:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 207/619] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:03:58 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 1568 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=9 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=32,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:03:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:03:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4ec7faa6-f4d2-4762-9245-b72ac3af2775 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.137992 Apr 12 02:03:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 203/620] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:03:59 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-interface => generated 282 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] Creating event network-changed:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:04:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4746890a-46c1-4d6a-b9a2-34ec2b7b1f42 req-0bab2497-0855-4721-ba12-376565b71f52 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053114 Apr 12 02:04:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 211/621] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:00 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:00 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lazy-loading 'services' on Instance uuid 84b4902c-24d9-4e96-b51f-e93e913aa681 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:04:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-85d7edce-1b34-4ae7-844c-7c4a33cd7752 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments/f6f462d6-7d7c-4843-97c3-d0ded790997f" status: 202 len: 0 microversion: 2.1 time: 0.189710 Apr 12 02:04:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 208/622] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:04:00 2023] DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments/f6f462d6-7d7c-4843-97c3-d0ded790997f => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f0390178-c83a-495e-8d57-bcf5674c933a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.091105 Apr 12 02:04:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 204/623] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:04:00 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] Creating event network-changed:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:04:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-fd26b85b-8938-4dec-b0cf-4c03c77c58cd req-e63e0360-74e7-4223-bc0a-a8c9bc677f32 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033118 Apr 12 02:04:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 212/624] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:02 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:04:07 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 HTTP/1.1 Apr 12 02:04:07 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:04:07 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:04:07 user devstack@n-api.service[72507]: Connection: close Apr 12 02:04:07 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:04:07 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:04:07 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:04:07 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:04:07 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72507]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72507]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:04:07 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72507]: method: > Apr 12 02:04:07 user devstack@n-api.service[72507]: exception: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72507]: action_args: {'id': 'c712f2db-72cb-4136-a719-56057d82fff6'} Apr 12 02:04:07 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:04:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-306589cc-457e-4fa3-b440-a3447074af01 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 404 len: 111 microversion: 2.1 time: 0.032808 Apr 12 02:04:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 209/625] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:07 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:04:07 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments HTTP/1.1 Apr 12 02:04:07 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:04:07 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:04:07 user devstack@n-api.service[72505]: Connection: close Apr 12 02:04:07 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:04:07 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:04:07 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:04:07 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:04:07 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72505]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72505]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:04:07 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72505]: method: > Apr 12 02:04:07 user devstack@n-api.service[72505]: exception: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72505]: action_args: {'server_id': 'c712f2db-72cb-4136-a719-56057d82fff6'} Apr 12 02:04:07 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:04:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4a9f3891-cd78-4955-a7f2-62bef854bc49 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.035373 Apr 12 02:04:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 205/626] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:04:07 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6/os-volume_attachments => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:04:07 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 HTTP/1.1 Apr 12 02:04:07 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:04:07 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:04:07 user devstack@n-api.service[72506]: Connection: close Apr 12 02:04:07 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:04:07 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:04:07 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:04:07 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:04:07 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:04:07 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:04:07 user devstack@n-api.service[72506]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:04:07 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:04:07 user devstack@n-api.service[72506]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:04:07 user devstack@n-api.service[72506]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:04:07 user devstack@n-api.service[72506]: method: > Apr 12 02:04:07 user devstack@n-api.service[72506]: exception: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72506]: action_args: {'id': 'c712f2db-72cb-4136-a719-56057d82fff6'} Apr 12 02:04:07 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:04:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:04:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:04:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bfa32b3c-ca43-4133-b5d9-3d866fee7d26 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 404 len: 111 microversion: 2.1 time: 0.038308 Apr 12 02:04:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 213/627] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:07 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:04:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-312717327", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "16f5655f-6169-43e9-9110-9fcbab7a3d5d"}], "security_groups": [{"name": "tempest-securitygroup--1634025199"}], "key_name": "tempest-keypair-1931079911", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:08 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] validate_networks() for [('16f5655f-6169-43e9-9110-9fcbab7a3d5d', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:04:09 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:04:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e84f076f-2387-4e8c-85f3-3ddcc00acf65 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 1.119791 Apr 12 02:04:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 210/628] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:04:08 2023] POST /compute/v2.1/servers => generated 404 bytes in 1122 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Apr 12 02:04:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-459c9f6c-312d-41d3-8ba8-c96d5ae9d2d2 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-459c9f6c-312d-41d3-8ba8-c96d5ae9d2d2 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:09 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-459c9f6c-312d-41d3-8ba8-c96d5ae9d2d2 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-459c9f6c-312d-41d3-8ba8-c96d5ae9d2d2 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 200 len: 1232 microversion: 2.1 time: 0.107204 Apr 12 02:04:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 206/629] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:09 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 1232 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:04:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] Creating event network-changed:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:04:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-0cc10c10-711e-4cb2-baed-56f6bf76ac96 req-fbab2281-0632-45a8-9cba-d67bd8f2301a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039957 Apr 12 02:04:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 214/630] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:11 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "name": "network-vif-plugged", "status": "completed", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] Creating event network-vif-plugged:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:04:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-6c7508a3-d977-436d-ab8c-36ae11ff4a50 req-cd5c2c13-7119-40bb-bd61-89968ea66fa8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042679 Apr 12 02:04:13 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 211/631] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:13 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "name": "network-vif-plugged", "status": "completed", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] Creating event network-vif-plugged:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:04:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7c53177d-aec1-40cc-b952-926d6ec8767d req-0044e951-a946-4f39-91b5-47f544d85e9a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032491 Apr 12 02:04:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 207/632] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:15 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:16 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b2bb1ed0-b5ee-4d7e-86d9-a412311cf33a tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1550 microversion: 2.1 time: 0.186946 Apr 12 02:04:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 215/633] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:16 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1550 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=23,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:04:25 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 HTTP/1.1 Apr 12 02:04:25 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:04:25 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:04:25 user devstack@n-api.service[72507]: Connection: close Apr 12 02:04:25 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:04:25 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:04:25 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:04:25 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:04:25 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:04:25 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:04:25 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:04:25 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:04:25 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:04:25 user devstack@n-api.service[72507]: method: > Apr 12 02:04:25 user devstack@n-api.service[72507]: exception: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:04:25 user devstack@n-api.service[72507]: action_args: {'id': '0fa3c7e6-a1a9-490e-ae02-8b1359736485'} Apr 12 02:04:25 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:04:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:04:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:04:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1961436b-f446-4d0b-bca1-c9607fca7400 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 404 len: 111 microversion: 2.1 time: 0.049604 Apr 12 02:04:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 212/634] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:25 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:04:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0ee7441d-e91b-4921-9f61-d80f09a22f0c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '.version_select at 0x7fb203041fc0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:25 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0ee7441d-e91b-4921-9f61-d80f09a22f0c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1196686774" status: 202 len: 0 microversion: 2.1 time: 0.024300 Apr 12 02:04:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 208/635] 10.0.0.210 () {62 vars in 1298 bytes} [Wed Apr 12 02:04:25 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1196686774 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:26 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lazy-loading 'services' on Instance uuid 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:04:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8635dbc2-4332-45d7-8ad3-41e04c435d2f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments/7353ae0d-d613-4966-adae-68aa36212f56" status: 202 len: 0 microversion: 2.1 time: 0.184446 Apr 12 02:04:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 216/636] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:04:26 2023] DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments/7353ae0d-d613-4966-adae-68aa36212f56 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:26 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:26 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-af7fa953-aa81-487a-9575-be9aed92b2f7 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052702 Apr 12 02:04:26 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 213/637] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:04:26 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1161614728", "imageRef": "", "flavorRef": "42", "block_device_mapping_v2": [{"uuid": "3e34e9fe-8906-4f7c-a3a9-c39ef07f0267", "source_type": "snapshot", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=15,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:04:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='3e34e9fe-8906-4f7c-a3a9-c39ef07f0267',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:04:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1b11e0f7-b021-4a15-a385-e0c129ac76d9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.771838 Apr 12 02:04:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 209/638] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:04:46 2023] POST /compute/v2.1/servers => generated 378 bytes in 773 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:04:46 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=13 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:46 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=24,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-34256bc2-b0e4-474b-8721-92a630767481 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-34256bc2-b0e4-474b-8721-92a630767481 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:46 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-34256bc2-b0e4-474b-8721-92a630767481 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-34256bc2-b0e4-474b-8721-92a630767481 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 200 len: 1052 microversion: 2.1 time: 0.070284 Apr 12 02:04:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 217/639] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:46 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 1052 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] Creating event network-changed:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:04:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-da33be86-2e17-4d8c-95ed-6f4889c7dac9 req-b5919c50-70d8-42b3-9aa7-df4f87f6ce7d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033226 Apr 12 02:04:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 214/640] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:49 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-541332af-119c-4575-8a09-b6c37af350b0 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043770 Apr 12 02:04:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 210/641] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:04:51 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681/os-volume_attachments => generated 25 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:04:51 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-22e4a434-221d-4488-8661-98cefdf348cf tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 204 len: 0 microversion: 2.1 time: 0.114757 Apr 12 02:04:51 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 218/642] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:04:51 2023] DELETE /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:51 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-795dd118-1ddd-4a8a-8880-772282dd150c tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 200 len: 1679 microversion: 2.1 time: 0.289251 Apr 12 02:04:51 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 215/643] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:51 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 1679 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-unplugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] Creating event network-vif-unplugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:04:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-82c9fcc0-f257-4165-8dbf-c52005315c03 req-5945b5d7-d51d-40e1-b564-4972efd75dbd service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.028008 Apr 12 02:04:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 211/644] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:51 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-deleted", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] Creating event network-vif-deleted:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:04:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-bb3bc082-2e7c-458a-ae43-2ef3d4cbda3f req-6a0d914d-025d-4c68-8832-e7e975cd2148 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045490 Apr 12 02:04:53 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 219/645] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84b4902c-24d9-4e96-b51f-e93e913aa681", "name": "network-vif-plugged", "status": "completed", "tag": "b1709cc0-5e25-4f89-aa42-8bc0a424adc1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:04:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] Creating event network-vif-plugged:b1709cc0-5e25-4f89-aa42-8bc0a424adc1 for instance 84b4902c-24d9-4e96-b51f-e93e913aa681 on user Apr 12 02:04:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-f4746aaa-c475-46ec-930f-147787bda325 req-9d5b5e69-c65e-494e-942a-65cd747fbb52 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.027940 Apr 12 02:04:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 216/646] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:04:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '34ca83258ae04dee81b8552a6c7e3c2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '910948b97a01432fb70094eb6ec30379', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c89b6aca-30fe-449c-bad5-0520243ea2ca tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 200 len: 1568 microversion: 2.1 time: 0.199747 Apr 12 02:04:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 212/647] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:58 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 1568 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:04:59 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:04:59 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7552f83c-db03-4cf2-a4eb-7c18b671a711 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 200 len: 1538 microversion: 2.1 time: 0.167993 Apr 12 02:04:59 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 220/648] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:04:59 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 1538 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:05:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "name": "network-vif-plugged", "status": "completed", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] Creating event network-vif-plugged:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:05:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-0756d54c-26c3-43a4-b2b1-0687d6180810 req-3052f83f-53cb-49f1-ad8d-01a732e49afd service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031209 Apr 12 02:05:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 217/649] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:01 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] Creating event network-changed:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:05:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-102465ce-d47c-498c-accc-992a00925b2e req-293e9cfe-7107-48ef-b0b1-0ce9c0d021a8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035251 Apr 12 02:05:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 213/650] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:02 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "346aa006-77dd-40bf-8a85-084d40c55ffe"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e03670ad-4da7-4ca6-9657-28d2d041a4fa tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.304769 Apr 12 02:05:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 221/651] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:05:02 2023] POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 194 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "name": "network-vif-plugged", "status": "completed", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] Creating event network-vif-plugged:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:05:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b68a2919-8abc-400e-95b1-ee5ac54cbae7 req-0523c4b7-3da6-4f75-9215-6180c97c7f08 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028948 Apr 12 02:05:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 218/652] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:03 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=18 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d2fd8787-bb05-4c16-b38c-a77b0d1f47c0 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.146995 Apr 12 02:05:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 214/653] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:05:16 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd/os-volume_attachments => generated 25 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=34,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=14 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:05:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4cca24c9-49e5-44b1-957d-5e5a45460656 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 204 len: 0 microversion: 2.1 time: 0.133795 Apr 12 02:05:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 219/654] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:05:16 2023] DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:16 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lazy-loading 'services' on Instance uuid 101410fb-7682-46d8-baf1-18fcebf81cb8 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:05:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-82209738-2a7f-458c-adf9-ec971a00201b tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/346aa006-77dd-40bf-8a85-084d40c55ffe" status: 202 len: 0 microversion: 2.1 time: 0.267128 Apr 12 02:05:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 222/655] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:05:16 2023] DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/346aa006-77dd-40bf-8a85-084d40c55ffe => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-ff1e26ee-d3c5-483c-b9c9-9c00b0a71ca2 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056248 Apr 12 02:05:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 220/656] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:05:16 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:05:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4e18c979-1117-4cf2-97ad-3737c0b22c01 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 200 len: 1665 microversion: 2.1 time: 0.294936 Apr 12 02:05:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 215/657] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:05:16 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 1665 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "name": "network-vif-unplugged", "status": "completed", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] Creating event network-vif-unplugged:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 02:05:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-1fef44ed-61a7-4b61-9493-a6988dfb072d req-d27579a5-6cf2-490e-b3e3-fa4e8d353c86 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030502 Apr 12 02:05:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 223/658] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:17 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:05:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] Creating event network-changed:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:05:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-4008f847-acb6-402a-b77a-1b535feab6f5 req-3dd1d418-93cc-4d20-a320-546d58facd70 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084576 Apr 12 02:05:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 216/659] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:17 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e9cd6447-47ba-42f1-bf3b-ee7b579df136 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 204 len: 0 microversion: 2.1 time: 0.127178 Apr 12 02:05:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 221/660] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:05:17 2023] DELETE /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:18 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5108c5bde1e249a2934f3e47496b1027', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '97353e0d7e084a1095d6ae46416d5660', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:05:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-ac9a392a-17bc-40b9-bf35-d18145229e88 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 200 len: 1556 microversion: 2.1 time: 0.307881 Apr 12 02:05:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 224/661] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:05:17 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 1556 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:05:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "name": "network-vif-plugged", "status": "completed", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}, {"server_uuid": "92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd", "name": "network-vif-deleted", "tag": "bc0f00ae-1855-439a-a945-9cc72e70fcc4"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Creating event network-vif-plugged:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 02:05:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] Creating event network-vif-deleted:bc0f00ae-1855-439a-a945-9cc72e70fcc4 for instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd on user Apr 12 02:05:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-24e94151-4f99-417c-8081-3b9486265966 req-c28e5ece-2cbe-4bb0-b30e-b4d610b2435e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.034345 Apr 12 02:05:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 217/662] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:19 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-unplugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}, {"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-plugged", "status": "completed", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Creating event network-vif-unplugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:05:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] Creating event network-vif-plugged:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:05:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-99e44dd3-521c-4d6e-bbc2-c3243c9f742c req-bb821d62-3fa1-4e4d-8729-e7234f4c9fac service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.028421 Apr 12 02:05:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 222/663] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:19 2023] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8c39c58-73b3-4574-9fcf-0c5901a81aa0", "name": "network-vif-deleted", "tag": "b64bd49d-20d0-4f9d-b07c-078a55b0542f"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] Creating event network-vif-deleted:b64bd49d-20d0-4f9d-b07c-078a55b0542f for instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 on user Apr 12 02:05:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-801b598d-ff97-4075-b977-a019220edeb3 req-d99ecbc5-b974-4afc-895f-b8befb95fca6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040791 Apr 12 02:05:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 225/664] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:21 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=16 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:05:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-80d70af2-b802-483d-af9b-be7f906593c0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 200 len: 1379 microversion: 2.1 time: 0.158053 Apr 12 02:05:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 218/665] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:05:36 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 1379 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: 84b4902c-24d9-4e96-b51f-e93e913aa681] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] HTTP exception thrown: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Request method failure captured: Apr 12 02:05:41 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 HTTP/1.1 Apr 12 02:05:41 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:05:41 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:05:41 user devstack@n-api.service[72507]: Connection: close Apr 12 02:05:41 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:05:41 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:05:41 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:05:41 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:05:41 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg44PjYOKvn626k4LU-GdnrYvKUw0JPm986bERmaP8HWBpoYllBF-zT9F33eDRWVPNLXGnPnqSXLqxvxPxbpnKT-DsqjWHFcOAkLCgQVhgVNvj2O847gDgRAlXz1fgsLlBnnV14mxHU4brxBUpZaN6c1uNs-kqCIbkSyeXxWQGBSWz0 Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Project-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Tenant-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:05:41 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:05:41 user devstack@n-api.service[72507]: X-User: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:05:41 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:05:41 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:05:41 user devstack@n-api.service[72507]: X-User-Id: 889189da060e4b54a48ce9a013e04a3d Apr 12 02:05:41 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:05:41 user devstack@n-api.service[72507]: method: > Apr 12 02:05:41 user devstack@n-api.service[72507]: exception: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. Apr 12 02:05:41 user devstack@n-api.service[72507]: action_args: {'id': '84b4902c-24d9-4e96-b51f-e93e913aa681'} Apr 12 02:05:41 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. Apr 12 02:05:41 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:05:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Returning 404 to user: Instance 84b4902c-24d9-4e96-b51f-e93e913aa681 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:05:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-8ec9e8a8-af07-4b07-94aa-ee87ef589db4 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681" status: 404 len: 111 microversion: 2.1 time: 0.035116 Apr 12 02:05:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 223/666] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:05:41 2023] GET /compute/v2.1/servers/84b4902c-24d9-4e96-b51f-e93e913aa681 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:05:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=19,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7dac152b-5db8-41e8-9de9-ded4a59e1a30 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '.version_select at 0x7fb202fd4af0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7dac152b-5db8-41e8-9de9-ded4a59e1a30 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2119544665" status: 202 len: 0 microversion: 2.1 time: 0.027938 Apr 12 02:05:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 226/667] 10.0.0.210 () {62 vars in 1298 bytes} [Wed Apr 12 02:05:41 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2119544665 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:05:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5ff294fb-42c4-45cb-9e9f-18e3cf47d38d tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 204 len: 0 microversion: 2.1 time: 0.121405 Apr 12 02:05:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 219/668] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:05:46 2023] DELETE /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:47 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:05:47 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a446b24c-da7c-4266-808f-379b07fe1091 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 200 len: 1495 microversion: 2.1 time: 0.176714 Apr 12 02:05:47 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 224/669] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:05:46 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 1495 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:05:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-unplugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] Creating event network-vif-unplugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 02:05:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c64a466a-b64c-412a-9436-3e3d2abfee32 req-265b31e8-449a-4ed8-b11d-f317a09c7661 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.134151 Apr 12 02:05:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 227/670] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:47 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:05:48 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-deleted", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:48 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] Creating event network-vif-deleted:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 02:05:48 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7b408186-f356-49ef-b661-40cee92509fd req-a81e2865-d481-46f8-a35f-f88e69d9ba6a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098238 Apr 12 02:05:48 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 220/671] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:48 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5c21fac-9fd9-46f8-851e-d1727f5ab0b9", "name": "network-vif-plugged", "status": "completed", "tag": "7edd32c8-a71c-4552-bb85-a073a178f87d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:05:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] Creating event network-vif-plugged:7edd32c8-a71c-4552-bb85-a073a178f87d for instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 on user Apr 12 02:05:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-979aa95a-c72a-43bb-94b4-6e40d77b9c1c req-6ffde510-5ad3-4632-bb97-7f5923cf51c5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031743 Apr 12 02:05:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 225/672] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:05:49 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=5,nova_api:DELETE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:05:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:05:59 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b12a8ef7-eb86-46c0-8e25-7e480f72dbb3 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.235539 Apr 12 02:05:59 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 228/673] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:05:59 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82/os-interface => generated 282 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] Creating event network-changed:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:06:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c8edd239-a625-4c1b-a92f-f07a14ca621b req-6aa3c4a3-3aa0-4553-b2f4-409993fe6e23 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038357 Apr 12 02:06:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 221/674] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:00 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1038265517", "imageRef": "", "flavorRef": "42", "networks": [{"uuid": "16f5655f-6169-43e9-9110-9fcbab7a3d5d"}], "security_groups": [{"name": "tempest-securitygroup--1634025199"}], "key_name": "tempest-keypair-1931079911", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "block_device_mapping_v2": [{"uuid": "136ed599-e72b-4d77-ad6e-104c14c24194", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] validate_networks() for [('16f5655f-6169-43e9-9110-9fcbab7a3d5d', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:06:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='136ed599-e72b-4d77-ad6e-104c14c24194',volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:06:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a34860b2-273b-4074-81d6-301b70a999f9 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.700234 Apr 12 02:06:04 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 226/675] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:06:03 2023] POST /compute/v2.1/servers => generated 404 bytes in 702 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:06:04 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f551004f-61af-4139-83ec-7676492950be tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:04 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f551004f-61af-4139-83ec-7676492950be tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:04 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-f551004f-61af-4139-83ec-7676492950be tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f551004f-61af-4139-83ec-7676492950be tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1076 microversion: 2.1 time: 0.080313 Apr 12 02:06:04 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 229/676] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:04 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1076 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:06 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:06 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] Creating event network-changed:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:06:06 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-374f72bd-aaa1-440b-8df4-a4600ca9cdaa req-148f4b70-3c87-4c1f-8ae4-861fba0b5b00 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047535 Apr 12 02:06:06 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 222/677] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:06 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f3ecf2ee-13fb-42e5-a7d0-0cd92e6e1288 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049776 Apr 12 02:06:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 227/678] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:06:07 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:06:07 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd HTTP/1.1 Apr 12 02:06:07 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:06:07 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:06:07 user devstack@n-api.service[72506]: Connection: close Apr 12 02:06:07 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:06:07 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:06:07 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:06:07 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:06:07 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:06:07 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:06:07 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:06:07 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:06:07 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:06:07 user devstack@n-api.service[72506]: method: > Apr 12 02:06:07 user devstack@n-api.service[72506]: exception: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:06:07 user devstack@n-api.service[72506]: action_args: {'id': '92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd'} Apr 12 02:06:07 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:06:07 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:06:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c7482408-598b-46ba-a9a3-617be7043551 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 404 len: 111 microversion: 2.1 time: 0.033431 Apr 12 02:06:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 230/679] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:07 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:06:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b34cfa69-4bc3-49bf-8c63-fa9e65b04361 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '.version_select at 0x7fb2030931c0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b34cfa69-4bc3-49bf-8c63-fa9e65b04361 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-97228139" status: 202 len: 0 microversion: 2.1 time: 0.024605 Apr 12 02:06:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 223/680] 10.0.0.210 () {62 vars in 1292 bytes} [Wed Apr 12 02:06:07 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-97228139 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e6c9fb72-39c6-465b-b973-13872413e6cd tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: .version_select at 0x7fb20301fc70>, body: {"keypair": {"name": "tempest-keypair-697615423", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMcsML6e7K9uGCYUP4kt9wqcviwiqIOeLK8iydSZvfdudxeVEzsOqCjAyVKD2yM1mhoVoiEzSVXVeJk4zlr5bUmtwCS+FsbuW8X8MAkaRSN0nt1uVNJsz9hu0laZovkkCA=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e6c9fb72-39c6-465b-b973-13872413e6cd tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:06:07 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e6c9fb72-39c6-465b-b973-13872413e6cd tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:06:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e6c9fb72-39c6-465b-b973-13872413e6cd tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.043918 Apr 12 02:06:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 228/681] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:06:07 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] [instance: b8c39c58-73b3-4574-9fcf-0c5901a81aa0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] HTTP exception thrown: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Request method failure captured: Apr 12 02:06:08 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 HTTP/1.1 Apr 12 02:06:08 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:06:08 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:06:08 user devstack@n-api.service[72506]: Connection: close Apr 12 02:06:08 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:06:08 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:06:08 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:06:08 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:06:08 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCBdFzcjUDu_h47lYPqMNF1640fiD6nAc6M8luGeQ0zqI_demkLy5z5SHe9Vo7x7sS495p-HrHh-3kWfKKsAMnvSVQMHvLQQ1t0kWeUy_9iPt0dHLJYpuJm8VuYR38dZHnnUmUIxxjyXFRKjPa5f6_AJnV_AlW-8T13azm5gTxxOho Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Project-Id: 97353e0d7e084a1095d6ae46416d5660 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestShelveInstance-1070508789 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/97353e0d7e084a1095d6ae46416d5660"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/97353e0d7e084a1095d6ae46416d5660"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/97353e0d7e084a1095d6ae46416d5660"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Tenant: tempest-TestShelveInstance-1070508789 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Tenant-Id: 97353e0d7e084a1095d6ae46416d5660 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestShelveInstance-1070508789 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-User: tempest-TestShelveInstance-1070508789-project-member Apr 12 02:06:08 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:06:08 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:06:08 user devstack@n-api.service[72506]: X-User-Id: 5108c5bde1e249a2934f3e47496b1027 Apr 12 02:06:08 user devstack@n-api.service[72506]: X-User-Name: tempest-TestShelveInstance-1070508789-project-member Apr 12 02:06:08 user devstack@n-api.service[72506]: method: > Apr 12 02:06:08 user devstack@n-api.service[72506]: exception: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. Apr 12 02:06:08 user devstack@n-api.service[72506]: action_args: {'id': 'b8c39c58-73b3-4574-9fcf-0c5901a81aa0'} Apr 12 02:06:08 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. Apr 12 02:06:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:06:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Returning 404 to user: Instance b8c39c58-73b3-4574-9fcf-0c5901a81aa0 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:06:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b12da89a-05aa-40dc-84be-372638cfabc7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0" status: 404 len: 111 microversion: 2.1 time: 0.034364 Apr 12 02:06:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 231/682] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:08 2023] GET /compute/v2.1/servers/b8c39c58-73b3-4574-9fcf-0c5901a81aa0 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:06:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-eea65b13-1ca5-4c84-8c69-44b11ad83cb7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] Calling method '.version_select at 0x7fb202f256c0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-eea65b13-1ca5-4c84-8c69-44b11ad83cb7 tempest-TestShelveInstance-1070508789 tempest-TestShelveInstance-1070508789-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-269156000" status: 202 len: 0 microversion: 2.1 time: 0.034335 Apr 12 02:06:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 224/683] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:06:08 2023] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-269156000 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "44d8159f-7e74-4b7f-9355-a33c1e23d9fd"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6cbf244e-36df-4591-a235-47135c06c2c7 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.263737 Apr 12 02:06:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 229/684] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:06:08 2023] POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 194 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-480594431", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "0875dfb2-68e9-4a37-8295-74f038cfba9a"}], "security_groups": [{"name": "tempest-securitygroup--1909867336"}], "key_name": "tempest-keypair-697615423", "user_data": "}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:09 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] validate_networks() for [('0875dfb2-68e9-4a37-8295-74f038cfba9a', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:06:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:06:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4130470f-4960-4c93-9d11-e95360d6bc12 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.681540 Apr 12 02:06:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 232/685] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:06:09 2023] POST /compute/v2.1/servers => generated 400 bytes in 684 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Apr 12 02:06:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d5600b14-bbbb-472c-ab6e-ef22fe13cd8c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:10 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d5600b14-bbbb-472c-ab6e-ef22fe13cd8c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:10 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-d5600b14-bbbb-472c-ab6e-ef22fe13cd8c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d5600b14-bbbb-472c-ab6e-ef22fe13cd8c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 200 len: 1234 microversion: 2.1 time: 0.129128 Apr 12 02:06:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 225/686] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:10 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 1234 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:06:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-adcd4c51-008c-42f4-8be1-d4bab8e64356 req-031cbd5e-c624-42f9-81ce-f66be2e0da90 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035452 Apr 12 02:06:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 230/687] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:12 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:13 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] Creating event network-changed:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:06:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f0176f90-0134-47f0-ab8b-d9b089ef4ffa req-166b577d-9bfb-4e06-afbb-27a620f0e863 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032776 Apr 12 02:06:13 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 233/688] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:13 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:06:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a0bf4e2f-71d7-495c-bea1-2d82bcd89353 req-e8ec28a7-d477-4181-8301-07dc02ecbb0e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062268 Apr 12 02:06:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 226/689] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:16 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "name": "network-vif-plugged", "status": "completed", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] Creating event network-vif-plugged:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:06:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-62742d85-a3c3-412b-a7ab-2c8d1d8dd427 req-ef7eeb73-d638-4c8d-8660-c7c36dcb14f1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040685 Apr 12 02:06:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 231/690] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:17 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lazy-loading 'services' on Instance uuid 101410fb-7682-46d8-baf1-18fcebf81cb8 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:06:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4a2b7c53-e3c8-447f-8203-893adfbbd45f tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/44d8159f-7e74-4b7f-9355-a33c1e23d9fd" status: 202 len: 0 microversion: 2.1 time: 0.214249 Apr 12 02:06:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 234/691] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:06:17 2023] DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/44d8159f-7e74-4b7f-9355-a33c1e23d9fd => generated 0 bytes in 216 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-efe2c958-5f96-4324-bddd-8ead7be0a5c5 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060845 Apr 12 02:06:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 227/692] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:06:17 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:18 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "name": "network-vif-plugged", "status": "completed", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] Creating event network-vif-plugged:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:06:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-11a9c37d-5ad7-4bb1-ae17-976070b62c10 req-5bab0e83-301c-4dc8-8cc2-2a49ac0ac451 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037359 Apr 12 02:06:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 232/693] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=18,nova_api:DELETE=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=12 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-90f998d0-e0ed-466a-a99e-3242352e3f0c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 200 len: 1379 microversion: 2.1 time: 0.167580 Apr 12 02:06:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 235/694] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:37 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 1379 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=25,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: d5c21fac-9fd9-46f8-851e-d1727f5ab0b9] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] HTTP exception thrown: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Request method failure captured: Apr 12 02:06:37 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 HTTP/1.1 Apr 12 02:06:37 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:06:37 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:06:37 user devstack@n-api.service[72505]: Connection: close Apr 12 02:06:37 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:06:37 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:06:37 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:06:37 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:06:37 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg44PjYOKvn626k4LU-GdnrYvKUw0JPm986bERmaP8HWBpoYllBF-zT9F33eDRWVPNLXGnPnqSXLqxvxPxbpnKT-DsqjWHFcOAkLCgQVhgVNvj2O847gDgRAlXz1fgsLlBnnV14mxHU4brxBUpZaN6c1uNs-kqCIbkSyeXxWQGBSWz0 Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Project-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Tenant-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:06:37 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:06:37 user devstack@n-api.service[72505]: X-User: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:06:37 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:06:37 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:06:37 user devstack@n-api.service[72505]: X-User-Id: 889189da060e4b54a48ce9a013e04a3d Apr 12 02:06:37 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:06:37 user devstack@n-api.service[72505]: method: > Apr 12 02:06:37 user devstack@n-api.service[72505]: exception: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. Apr 12 02:06:37 user devstack@n-api.service[72505]: action_args: {'id': 'd5c21fac-9fd9-46f8-851e-d1727f5ab0b9'} Apr 12 02:06:37 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. Apr 12 02:06:37 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Returning 404 to user: Instance d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:06:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0687ac18-b34d-40e0-8e87-2044b75ebafa tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9" status: 404 len: 111 microversion: 2.1 time: 0.215980 Apr 12 02:06:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 228/695] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:37 2023] GET /compute/v2.1/servers/d5c21fac-9fd9-46f8-851e-d1727f5ab0b9 => generated 111 bytes in 217 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:06:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b69a061e-6367-41d0-808f-bd096f5d429a tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 204 len: 0 microversion: 2.1 time: 0.130952 Apr 12 02:06:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 236/696] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:06:37 2023] DELETE /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e79ece29-0f6d-470f-ba23-d28614e3a779 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 200 len: 1379 microversion: 2.1 time: 0.346309 Apr 12 02:06:37 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 233/697] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:37 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 1379 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '889189da060e4b54a48ce9a013e04a3d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8fed0886b2404f41a659dc6356771e9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8587a343-cd7e-4023-8b7c-7444b445f946 tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 200 len: 1509 microversion: 2.1 time: 0.223987 Apr 12 02:06:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 229/698] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:37 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 1509 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-unplugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] Creating event network-vif-unplugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 02:06:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-fe50a543-914a-4195-b731-3bd47a2746df req-7018cadb-f021-488c-a7f5-d077459c5ea4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029324 Apr 12 02:06:37 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 234/699] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:37 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5fbca1a7-d5ef-4404-87e7-b219f4ab7c2f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 204 len: 0 microversion: 2.1 time: 0.125080 Apr 12 02:06:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 237/700] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:06:37 2023] DELETE /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "name": "network-vif-unplugged", "status": "completed", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] Creating event network-vif-unplugged:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:06:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-627db1be-0c19-470a-a681-94266c972f24 req-4088b88b-66ce-49a1-b04c-38abaf3fe3b0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038998 Apr 12 02:06:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 235/701] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:38 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:38 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c9d35ffa-1217-462d-9f44-dbd8fdc1b521 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 200 len: 1385 microversion: 2.1 time: 0.407547 Apr 12 02:06:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 230/702] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:37 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 1385 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:06:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-plugged", "status": "completed", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}, {"server_uuid": "fdbd8add-ac0e-42ce-9b34-5878c6bec208", "name": "network-vif-deleted", "tag": "af0cfa6a-437f-4754-87f3-f5715490cbbf"}, {"server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "name": "network-vif-deleted", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Creating event network-vif-plugged:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 02:06:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Creating event network-vif-deleted:af0cfa6a-437f-4754-87f3-f5715490cbbf for instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 on user Apr 12 02:06:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] Creating event network-vif-deleted:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:06:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f59ddf3d-de30-4e97-8955-c56010cf0806 req-22ce353d-fb2b-4160-86a7-67e856aba4d5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.036705 Apr 12 02:06:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 238/703] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:39 2023] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b0bb153-7800-4298-a293-7a2b8cebb194", "name": "network-vif-plugged", "status": "completed", "tag": "0741e554-620f-442e-b359-4d2c83da491a"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:06:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] Creating event network-vif-plugged:0741e554-620f-442e-b359-4d2c83da491a for instance 2b0bb153-7800-4298-a293-7a2b8cebb194 on user Apr 12 02:06:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-cc93f823-2783-47f7-a0a4-66ae59c5bac1 req-057fc49a-a78a-4a1f-9570-5f518552e4b1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030177 Apr 12 02:06:40 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 236/704] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:06:40 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:06:54 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:06:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-669b576f-4314-4bf9-aa16-b5decf122134 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1428 microversion: 2.1 time: 0.151424 Apr 12 02:06:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 231/705] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:06:54 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1428 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=18,nova_api:UPDATE=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=38,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:00 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:07:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-addf39a8-6a35-42dd-9541-90dd71e54f20 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 200 len: 1544 microversion: 2.1 time: 0.156944 Apr 12 02:07:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 239/706] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:00 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 1544 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f04d08d8-3500-4147-8f47-b1c962b7ef35 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046677 Apr 12 02:07:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 237/707] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:07:07 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9ab4d808-467b-4974-b606-a8029e17b183"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-71789377-f12c-43cb-9af0-d73239d643fc tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.292303 Apr 12 02:07:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 232/708] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:07:10 2023] POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 194 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:17 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lazy-loading 'services' on Instance uuid 101410fb-7682-46d8-baf1-18fcebf81cb8 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:07:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e49c3e81-c720-4b3b-8dd9-efd83fb31000 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/9ab4d808-467b-4974-b606-a8029e17b183" status: 202 len: 0 microversion: 2.1 time: 0.168539 Apr 12 02:07:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 240/709] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:07:17 2023] DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/9ab4d808-467b-4974-b606-a8029e17b183 => generated 0 bytes in 170 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bcacc166-db21-4e96-a741-474323eabdb4 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046382 Apr 12 02:07:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 238/710] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:07:17 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] [instance: fdbd8add-ac0e-42ce-9b34-5878c6bec208] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] HTTP exception thrown: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Request method failure captured: Apr 12 02:07:27 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 HTTP/1.1 Apr 12 02:07:27 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:07:27 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:07:27 user devstack@n-api.service[72505]: Connection: close Apr 12 02:07:27 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:07:27 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:07:27 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:07:27 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:07:27 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg44PjYOKvn626k4LU-GdnrYvKUw0JPm986bERmaP8HWBpoYllBF-zT9F33eDRWVPNLXGnPnqSXLqxvxPxbpnKT-DsqjWHFcOAkLCgQVhgVNvj2O847gDgRAlXz1fgsLlBnnV14mxHU4brxBUpZaN6c1uNs-kqCIbkSyeXxWQGBSWz0 Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Project-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/8fed0886b2404f41a659dc6356771e9c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Tenant-Id: 8fed0886b2404f41a659dc6356771e9c Apr 12 02:07:27 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-502185932 Apr 12 02:07:27 user devstack@n-api.service[72505]: X-User: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:07:27 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:07:27 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:07:27 user devstack@n-api.service[72505]: X-User-Id: 889189da060e4b54a48ce9a013e04a3d Apr 12 02:07:27 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerRescueNegativeTestJSON-502185932-project-member Apr 12 02:07:27 user devstack@n-api.service[72505]: method: > Apr 12 02:07:27 user devstack@n-api.service[72505]: exception: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. Apr 12 02:07:27 user devstack@n-api.service[72505]: action_args: {'id': 'fdbd8add-ac0e-42ce-9b34-5878c6bec208'} Apr 12 02:07:27 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. Apr 12 02:07:27 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] Returning 404 to user: Instance fdbd8add-ac0e-42ce-9b34-5878c6bec208 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:07:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9e0438a7-c9e9-487c-89cc-84a3220426fb tempest-ServerRescueNegativeTestJSON-502185932 tempest-ServerRescueNegativeTestJSON-502185932-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208" status: 404 len: 111 microversion: 2.1 time: 0.038550 Apr 12 02:07:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 233/711] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:27 2023] GET /compute/v2.1/servers/fdbd8add-ac0e-42ce-9b34-5878c6bec208 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:07:28 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 HTTP/1.1 Apr 12 02:07:28 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:07:28 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:07:28 user devstack@n-api.service[72506]: Connection: close Apr 12 02:07:28 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:07:28 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:07:28 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:07:28 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:07:28 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72506]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72506]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:07:28 user devstack@n-api.service[72506]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72506]: method: > Apr 12 02:07:28 user devstack@n-api.service[72506]: exception: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72506]: action_args: {'id': '2b0bb153-7800-4298-a293-7a2b8cebb194'} Apr 12 02:07:28 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:07:28 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e478f85c-e57a-401f-aaf9-538bebcfb66f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 404 len: 111 microversion: 2.1 time: 0.036512 Apr 12 02:07:28 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 241/712] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:28 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:07:28 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 HTTP/1.1 Apr 12 02:07:28 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:07:28 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:07:28 user devstack@n-api.service[72507]: Connection: close Apr 12 02:07:28 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:07:28 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:07:28 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:07:28 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:07:28 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72507]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72507]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:07:28 user devstack@n-api.service[72507]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72507]: method: > Apr 12 02:07:28 user devstack@n-api.service[72507]: exception: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72507]: action_args: {'id': '2b0bb153-7800-4298-a293-7a2b8cebb194'} Apr 12 02:07:28 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:07:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6393d834-8d56-4fd9-ade8-bac7a4c28650 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 404 len: 111 microversion: 2.1 time: 0.040713 Apr 12 02:07:28 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 239/713] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:07:28 2023] DELETE /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2b0bb153-7800-4298-a293-7a2b8cebb194] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:07:28 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 HTTP/1.1 Apr 12 02:07:28 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:07:28 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:07:28 user devstack@n-api.service[72505]: Connection: close Apr 12 02:07:28 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:07:28 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:07:28 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:07:28 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:07:28 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:07:28 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:07:28 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:07:28 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:07:28 user devstack@n-api.service[72505]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:07:28 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:07:28 user devstack@n-api.service[72505]: method: > Apr 12 02:07:28 user devstack@n-api.service[72505]: exception: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72505]: action_args: {'id': '2b0bb153-7800-4298-a293-7a2b8cebb194'} Apr 12 02:07:28 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. Apr 12 02:07:28 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:07:28 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2b0bb153-7800-4298-a293-7a2b8cebb194 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:07:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1c763b06-668b-43f4-aa36-5f7ff4e8a01a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194" status: 404 len: 111 microversion: 2.1 time: 0.032666 Apr 12 02:07:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 234/714] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:28 2023] GET /compute/v2.1/servers/2b0bb153-7800-4298-a293-7a2b8cebb194 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-volume-backed-server-1293359077", "imageRef": "", "flavorRef": "42", "block_device_mapping_v2": [{"uuid": "a91e0965-aef8-40da-921c-cd9566b59a98", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:34 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:07:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:07:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:07:35 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a91e0965-aef8-40da-921c-cd9566b59a98',volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:07:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0323eeea-ce23-4468-ad8f-eb0ea0c399eb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.668118 Apr 12 02:07:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 242/715] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:07:34 2023] POST /compute/v2.1/servers => generated 378 bytes in 670 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:07:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b9d059ac-9c39-4a3d-9a00-8505ffe18d88 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:35 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-b9d059ac-9c39-4a3d-9a00-8505ffe18d88 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:35 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-b9d059ac-9c39-4a3d-9a00-8505ffe18d88 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:07:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b9d059ac-9c39-4a3d-9a00-8505ffe18d88 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 200 len: 1066 microversion: 2.1 time: 0.072845 Apr 12 02:07:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 240/716] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:35 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 1066 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:07:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:37 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] Creating event network-changed:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:07:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-0b4d6a62-d685-4c85-b90a-942da6a336f2 req-152f6d12-fd98-4d36-a9df-b6a1cd74916c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050622 Apr 12 02:07:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 235/717] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:37 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:44 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "name": "network-vif-plugged", "status": "completed", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] Creating event network-vif-plugged:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:07:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-26694122-9304-4f18-a775-ae722abaace1 req-e5f55646-5d50-4c34-b1e4-72dfc16a846a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042488 Apr 12 02:07:44 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 243/718] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:44 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "name": "network-vif-plugged", "status": "completed", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] Creating event network-vif-plugged:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:07:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-da3de871-e045-4087-8e37-f85bffa4649b req-15b3c14e-d19b-4c78-a402-3d465848a048 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037842 Apr 12 02:07:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 241/719] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:46 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5f95bf27-2269-43f5-8db5-82a451012fd4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.111133 Apr 12 02:07:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 236/720] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:07:54 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/os-interface => generated 282 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] Creating event network-changed:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:07:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-feaf7e49-3470-4480-83e8-1da4370ec780 req-8201aeda-5e57-466b-8dc4-30db7079b8e0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069177 Apr 12 02:07:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 244/721] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:56 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:56 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:07:56 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f59f6c25-c548-466a-8f86-8457021db490 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1428 microversion: 2.1 time: 0.151802 Apr 12 02:07:56 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 242/722] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:56 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1428 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "84"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Old instance type m1.nano, new instance type m1.micro {{(pid=72505) resize /opt/stack/nova/nova/compute/api.py:4223}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Getting quotas for user e8e8b9f0307b4d3fb54d89962798026b and project 117589cba4a54dd581f4ec5c26b4cd2a. Resources: {'cores', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:07:56 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:07:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2f761133-141f-44aa-93cc-06ae37434063 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action" status: 202 len: 0 microversion: 2.1 time: 0.328988 Apr 12 02:07:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 237/723] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:07:56 2023] POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action => generated 0 bytes in 331 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:56 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:07:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a5b2874f-d490-49cd-8195-6ca40e351c9e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1437 microversion: 2.1 time: 0.203776 Apr 12 02:07:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 245/724] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:07:56 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1437 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:07:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:58 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] Creating event network-changed:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:07:58 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-d57ff9ed-0bc3-4b70-b46a-ae827f5a5eaf req-dd230fc9-4d1a-4f6c-a221-737cc13bd647 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031603 Apr 12 02:07:58 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 243/725] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:58 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:07:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-unplugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:07:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:07:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:07:59 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:07:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Creating event network-vif-unplugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:07:59 user devstack@n-api.service[72505]: DEBUG nova.compute.api [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72505) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:07:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f451f0a8-cb58-45f5-b534-04b18b36fc76 req-aee0b37a-cfe1-438b-9f01-3dcaea1006d8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047116 Apr 12 02:07:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 238/726] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:07:59 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-3b30ef24-0759-4c0f-b006-1bf03fef143e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-interface" status: 200 len: 283 microversion: 2.1 time: 0.136432 Apr 12 02:08:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 246/727] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:08:00 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-interface => generated 283 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] Creating event network-changed:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:08:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-a4349793-c133-4731-ae7a-40960e6f3deb req-11141171-4e9a-489f-bf8a-53e08b6cb2ac service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037207 Apr 12 02:08:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 244/728] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:08:01 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-76737dc6-44ca-4bae-a26c-e0cfe7349c1f tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/ips" status: 200 len: 108 microversion: 2.1 time: 0.043651 Apr 12 02:08:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 239/729] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:08:01 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/ips => generated 108 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:01 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:08:01 user devstack@n-api.service[72506]: DEBUG nova.compute.api [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72506) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:08:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-cf300519-91ca-4172-b744-1093be180b7c req-a7e9cfc1-82c8-43fc-ad36-f2389904ee87 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067750 Apr 12 02:08:01 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 247/730] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:08:01 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ee436c5d-23fd-481d-b69f-9563a02987ee"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a33b4aa8-f791-42b7-8214-48c86bf61ef5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.298420 Apr 12 02:08:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 245/731] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:08:03 2023] POST /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments => generated 194 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-52daddfe-91e2-4da4-a2a0-464b3b5579a8 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.238868 Apr 12 02:08:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 240/732] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:08:07 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 25 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "33e9c9ea-ca01-4cb4-8946-2e6bd5cdcc7a"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-2a71a0ec-90d7-476f-bd83-6ad0293dc53c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.361768 Apr 12 02:08:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 248/733] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:08:10 2023] POST /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 194 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ee436c5d-23fd-481d-b69f-9563a02987ee"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:08:15 user devstack@n-api.service[72507]: request: POST /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments HTTP/1.1 Apr 12 02:08:15 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:08:15 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:08:15 user devstack@n-api.service[72507]: Connection: close Apr 12 02:08:15 user devstack@n-api.service[72507]: Content-Length: 74 Apr 12 02:08:15 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:08:15 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:08:15 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:08:15 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:08:15 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:08:15 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:08:15 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:08:15 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:08:15 user devstack@n-api.service[72507]: Apr 12 02:08:15 user devstack@n-api.service[72507]: {"volumeAttachment": {"volumeId": "ee436c5d-23fd-481d-b69f-9563a02987ee"}} Apr 12 02:08:15 user devstack@n-api.service[72507]: method: > Apr 12 02:08:15 user devstack@n-api.service[72507]: exception: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 Apr 12 02:08:15 user devstack@n-api.service[72507]: action_args: {'server_id': '34a07ab6-379e-48d9-98ae-c32981534331', 'body': {'volumeAttachment': {'volumeId': 'ee436c5d-23fd-481d-b69f-9563a02987ee'}}} Apr 12 02:08:15 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5139, in attach_volume Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4979, in _check_volume_already_attached Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 Apr 12 02:08:15 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:08:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 400 to user: Invalid volume: volume ee436c5d-23fd-481d-b69f-9563a02987ee is already attached to instances: 34a07ab6-379e-48d9-98ae-c32981534331 {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:08:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6f62861e-0473-4a90-b297-454c18f04d91 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.114297 Apr 12 02:08:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 246/734] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:08:15 2023] POST /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments => generated 174 bytes in 116 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lazy-loading 'services' on Instance uuid 34a07ab6-379e-48d9-98ae-c32981534331 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:08:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a691e44f-7e72-4725-b671-0e5607060523 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments/ee436c5d-23fd-481d-b69f-9563a02987ee" status: 202 len: 0 microversion: 2.1 time: 0.197495 Apr 12 02:08:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 241/735] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:08:15 2023] DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments/ee436c5d-23fd-481d-b69f-9563a02987ee => generated 0 bytes in 199 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-959838ec-6609-4eb8-8594-e61425db7015 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047739 Apr 12 02:08:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 249/736] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:08:15 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:08:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72507) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:08:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7d0b6918-11da-4e75-b6e3-77fd178a1434 req-e399c43e-1c62-4a3d-a43f-657ab1a06c33 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045155 Apr 12 02:08:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 247/737] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:08:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:18 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:08:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:08:18 user devstack@n-api.service[72505]: DEBUG nova.compute.api [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 is migrating within the same cell, copying events to all relevant hosts: {'user'} {{(pid=72505) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5948}} Apr 12 02:08:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a1dc46d3-048a-46f8-8aa3-47d0f23b08dc req-d9a7f401-0225-4e56-b523-a1c02e6decbe service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044414 Apr 12 02:08:18 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 242/738] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:08:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:18 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lazy-loading 'services' on Instance uuid 101410fb-7682-46d8-baf1-18fcebf81cb8 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:08:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e30cb6f4-b362-41f7-bde9-fe175e83f9be tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/33e9c9ea-ca01-4cb4-8946-2e6bd5cdcc7a" status: 202 len: 0 microversion: 2.1 time: 0.189554 Apr 12 02:08:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 250/739] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:08:18 2023] DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments/33e9c9ea-ca01-4cb4-8946-2e6bd5cdcc7a => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-893ba0ee-28b1-4d49-a4f6-c4bd21bc2d24 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067462 Apr 12 02:08:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 248/740] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:08:19 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=27 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:25 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:08:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e3993df6-c0b1-42b0-955a-1117f88c0eb2 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 200 len: 1393 microversion: 2.1 time: 0.166437 Apr 12 02:08:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 243/741] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:08:25 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 1393 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=14 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:08:47 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:08:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-fe72fe2e-528c-4247-89bb-26dd19aaaedc tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1554 microversion: 2.1 time: 0.156042 Apr 12 02:08:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 251/742] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:08:47 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1554 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=16 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:05 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-c1a463b6-9301-46fe-8d29-40b943ae2e36 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046971 Apr 12 02:09:05 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 249/743] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:09:05 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=5 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:05 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:09:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0a525f07-5445-4d0a-a91f-5c4a796d99d6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 204 len: 0 microversion: 2.1 time: 0.116379 Apr 12 02:09:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 244/744] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:09:05 2023] DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:06 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:06 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-830005f4-4234-4a57-ac7f-f0b30a3e7a4d tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 200 len: 1668 microversion: 2.1 time: 0.184073 Apr 12 02:09:06 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 252/745] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:05 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 1668 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:06 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "name": "network-vif-unplugged", "status": "completed", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:06 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] Creating event network-vif-unplugged:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:09:06 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-6b7369f3-97da-49cb-936a-87bf82657fc3 req-58f9377a-9f49-4e4e-aefa-cefcc5a2dc96 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029716 Apr 12 02:09:06 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 250/746] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:06 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "name": "network-vif-deleted", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] Creating event network-vif-deleted:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:09:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f0c4a150-7e1f-4da2-a6dc-b23d651790d0 req-af1b03d8-395c-4797-b243-1864317a4877 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028859 Apr 12 02:09:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 245/747] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:07 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34a07ab6-379e-48d9-98ae-c32981534331", "name": "network-vif-plugged", "status": "completed", "tag": "b7002ebc-047e-4856-bf38-9398ce8b8e6b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] Creating event network-vif-plugged:b7002ebc-047e-4856-bf38-9398ce8b8e6b for instance 34a07ab6-379e-48d9-98ae-c32981534331 on user Apr 12 02:09:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-6229bd3d-7729-4f04-8b6f-97343239afce req-9a56edd4-5d36-4819-9714-b206afe6cbdd service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037007 Apr 12 02:09:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 253/748] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:08 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a552d0ca-94c4-4f0e-a01b-4d60a489da97 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.042046 Apr 12 02:09:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 251/749] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:09:09 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] Creating event network-changed:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:09:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7c6830e6-9471-49d5-842f-abd9c5dd94c5 req-2134622c-dbc7-4a4a-9e05-081683dc0f2b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040189 Apr 12 02:09:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 246/750] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:19 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:21 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:09:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-927c3da7-fb6f-4f83-baea-9f79984dd84c tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 204 len: 0 microversion: 2.1 time: 0.111866 Apr 12 02:09:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 254/751] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:09:21 2023] DELETE /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=14 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:21 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '34ca83258ae04dee81b8552a6c7e3c2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '910948b97a01432fb70094eb6ec30379', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d3cc4a50-5bc8-4d94-836f-e787d7b29196 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 200 len: 1574 microversion: 2.1 time: 0.200438 Apr 12 02:09:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 252/752] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:21 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 1574 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "name": "network-vif-unplugged", "status": "completed", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] Creating event network-vif-unplugged:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:09:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-6d15c67c-3c78-42ce-8bdd-9337b09e4506 req-7a4dc4b9-03a1-4a03-9cc7-4f81affeb49e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033878 Apr 12 02:09:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 247/753] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:21 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "name": "network-vif-plugged", "status": "completed", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}, {"server_uuid": "101410fb-7682-46d8-baf1-18fcebf81cb8", "name": "network-vif-deleted", "tag": "280a0eb1-7bff-42d2-a75e-69eba211ffbb"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Creating event network-vif-plugged:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:09:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] Creating event network-vif-deleted:280a0eb1-7bff-42d2-a75e-69eba211ffbb for instance 101410fb-7682-46d8-baf1-18fcebf81cb8 on user Apr 12 02:09:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-b45f06b1-db22-4679-87ea-22221b7821f0 req-8002d912-c6d3-47a1-834d-297e2106424c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.119899 Apr 12 02:09:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 255/754] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:23 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:25 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2b23ba6d-5db4-47eb-9728-c70dc4ef20ef tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 200 len: 1393 microversion: 2.1 time: 0.156631 Apr 12 02:09:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 253/755] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:25 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 1393 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestVolumeBootPatternsnapshot-102112020"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:25 user devstack@n-api.service[72505]: INFO nova.compute.api [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Attempting to quiesce instance before volume snapshot. Apr 12 02:09:25 user devstack@n-api.service[72505]: INFO nova.compute.api [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Skipping quiescing instance: QEMU guest agent is not enabled. Apr 12 02:09:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Creating snapshot from volume a91e0965-aef8-40da-921c-cd9566b59a98. {{(pid=72505) snapshot_instance /opt/stack/nova/nova/compute/api.py:3511}} Apr 12 02:09:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-adc8e737-cf20-4874-abeb-fa7dd34f1d92 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c/action" status: 202 len: 0 microversion: 2.1 time: 2.451391 Apr 12 02:09:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 248/756] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:09:25 2023] POST /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c/action => generated 0 bytes in 2454 msecs (HTTP/1.1 202) 10 headers in 449 bytes (1 switches on core 0) Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-image-snapshot-server-2306241", "imageRef": "594e2609-87fd-4742-9919-8292fb3c9aa4", "flavorRef": "42", "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:09:28 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name='/dev/vda',device_type='disk',disk_bus='virtio',encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='d6c1c4cc-af38-437e-a229-fef7a5bfeebf',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:09:28 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b612baab-26a6-410a-86d3-aad5ec829514 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.599998 Apr 12 02:09:28 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 256/757] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:09:28 2023] POST /compute/v2.1/servers => generated 378 bytes in 604 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:09:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0d5b9945-84c2-41e9-acaf-cf81e4014ac6 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0d5b9945-84c2-41e9-acaf-cf81e4014ac6 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:29 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0d5b9945-84c2-41e9-acaf-cf81e4014ac6 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0d5b9945-84c2-41e9-acaf-cf81e4014ac6 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 200 len: 1221 microversion: 2.1 time: 0.065666 Apr 12 02:09:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 254/758] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:29 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 1221 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] Creating event network-changed:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:09:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-33057fd6-77f2-49d0-85d4-c4143666ad5a req-5a107e9f-b8ca-451d-9562-2a13551f67d0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032491 Apr 12 02:09:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 249/759] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:31 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "name": "network-vif-plugged", "status": "completed", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] Creating event network-vif-plugged:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:09:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-6e7e2589-2f98-4a4b-899b-7787612b9fba req-9f3ea644-4f09-4012-b12c-ce2acdb8eabd service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050089 Apr 12 02:09:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 257/760] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:43 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "name": "network-vif-plugged", "status": "completed", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:45 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] Creating event network-vif-plugged:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:09:45 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-8ba2a752-51dd-49c9-a403-89510813d2ae req-49101597-c343-472e-8c3f-3ed0d34b493b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040363 Apr 12 02:09:45 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 255/761] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:09:45 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=20,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-228a0cb7-4752-4207-9fb9-8b2528d4f78d tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action" status: 204 len: 0 microversion: 2.1 time: 0.091504 Apr 12 02:09:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 250/762] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:09:47 2023] POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action => generated 0 bytes in 94 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:47 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:47 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4f22d793-d1c4-4433-9bf7-d07a736edc1f tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1554 microversion: 2.1 time: 0.204034 Apr 12 02:09:47 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 258/763] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:47 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1554 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:56 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:09:56 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 HTTP/1.1 Apr 12 02:09:56 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:09:56 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:09:56 user devstack@n-api.service[72507]: Connection: close Apr 12 02:09:56 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:09:56 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:09:56 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:09:56 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:09:56 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:09:56 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:09:56 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:09:56 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:09:56 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:09:56 user devstack@n-api.service[72507]: method: > Apr 12 02:09:56 user devstack@n-api.service[72507]: exception: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:09:56 user devstack@n-api.service[72507]: action_args: {'id': '34a07ab6-379e-48d9-98ae-c32981534331'} Apr 12 02:09:56 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:09:56 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:09:56 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:09:56 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e9374fc9-14a9-4916-8ae7-6734d71d0f26 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 404 len: 111 microversion: 2.1 time: 0.036643 Apr 12 02:09:56 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 256/764] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:56 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:09:56 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-058b4b47-73db-4003-ac35-89324fe82c22 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '.version_select at 0x7fb202e68820>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:56 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-058b4b47-73db-4003-ac35-89324fe82c22 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-697615423" status: 202 len: 0 microversion: 2.1 time: 0.025889 Apr 12 02:09:56 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 251/765] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:09:56 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-697615423 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:09:57 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c3ac0d58-b9cc-414c-b477-5d738746b598 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: .version_select at 0x7fb203152c20>, body: {"keypair": {"name": "tempest-keypair-725616642", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEnGvkYnZyorUiJqxXvdxoifFTG6MQo2aX6QPuwm2S6OSg8q2DyZRgIWfNNU7KxQdTj8WezhTcNpj4o5/d6t4zoRglpDW6jCf0t5khDjiNWkWkgste5dsdaPFBGYxqW8AQ=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:57 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-c3ac0d58-b9cc-414c-b477-5d738746b598 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:09:57 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-c3ac0d58-b9cc-414c-b477-5d738746b598 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:09:57 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c3ac0d58-b9cc-414c-b477-5d738746b598 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.051513 Apr 12 02:09:57 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 259/766] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:09:56 2023] POST /compute/v2.1/os-keypairs => generated 385 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:09:58 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1253726085", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "0875dfb2-68e9-4a37-8295-74f038cfba9a"}], "security_groups": [{"name": "tempest-securitygroup--2002138467"}], "key_name": "tempest-keypair-725616642", "user_data": "}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:09:58 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] validate_networks() for [('0875dfb2-68e9-4a37-8295-74f038cfba9a', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Getting quotas for user 189b7aba48014bedacfa021cc3cad5e4 and project 2acf278c981e481fbe43e415c76aa6e7. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:09:59 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:09:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-da4d6d48-f767-44dd-ab99-58cc787ba951 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.938886 Apr 12 02:09:59 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 257/767] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:09:58 2023] POST /compute/v2.1/servers => generated 400 bytes in 941 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:09:59 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:09:59 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-73bb46ed-b163-43f5-862a-53737964de85 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:09:59 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-73bb46ed-b163-43f5-862a-53737964de85 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:09:59 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-73bb46ed-b163-43f5-862a-53737964de85 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:09:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-73bb46ed-b163-43f5-862a-53737964de85 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 200 len: 1235 microversion: 2.1 time: 0.135953 Apr 12 02:09:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 252/768] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:09:59 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 1235 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:10:01 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:01 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:10:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] Creating event network-changed:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:10:01 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8d8e4acc-bb23-4222-aaae-d0e40c47c985 req-0df5c3df-0de6-435b-8f5b-deb1670f0173 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031023 Apr 12 02:10:01 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 260/769] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:10:01 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:10:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "name": "network-vif-plugged", "status": "completed", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:10:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] Creating event network-vif-plugged:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:10:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3ee8d5be-ea6b-4b0d-ad75-fcea58d6511b req-42e46f29-ac71-4a4a-8d95-3dca3ca753a8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034411 Apr 12 02:10:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 258/770] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:10:03 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:10:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "name": "network-vif-plugged", "status": "completed", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:10:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] Creating event network-vif-plugged:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:10:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c442920f-3ccf-4e5e-bcf0-449f872d45a0 req-abe5b4bd-702f-4e40-ab5c-a88d65edf495 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031516 Apr 12 02:10:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 253/771] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:10:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] [instance: 101410fb-7682-46d8-baf1-18fcebf81cb8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] HTTP exception thrown: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Request method failure captured: Apr 12 02:10:11 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 HTTP/1.1 Apr 12 02:10:11 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:10:11 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:10:11 user devstack@n-api.service[72506]: Connection: close Apr 12 02:10:11 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:10:11 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:10:11 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:10:11 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:10:11 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhFTkuQScednfSaENmIGjTO_nfYcNVKHvh45F2vgsSHDsTc2mX7MVNrKE1qrKH2HHhZPIjq3s9EtL5Pi7kM7ZcBE2IQKG0EJpa2HFD98Ze4FmlUi7QZEuFQw7k9Mdc0EpAZqLsrW8n7SHSoIHRUV8nVJTjy15Eo0pQrxeqPKrsdQVQ8 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Project-Id: 910948b97a01432fb70094eb6ec30379 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Project-Name: tempest-SnapshotDataIntegrityTests-1739495878 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/910948b97a01432fb70094eb6ec30379"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/910948b97a01432fb70094eb6ec30379"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/910948b97a01432fb70094eb6ec30379"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Tenant: tempest-SnapshotDataIntegrityTests-1739495878 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Tenant-Id: 910948b97a01432fb70094eb6ec30379 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-SnapshotDataIntegrityTests-1739495878 Apr 12 02:10:11 user devstack@n-api.service[72506]: X-User: tempest-SnapshotDataIntegrityTests-1739495878-project-member Apr 12 02:10:11 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:10:11 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:10:11 user devstack@n-api.service[72506]: X-User-Id: 34ca83258ae04dee81b8552a6c7e3c2f Apr 12 02:10:11 user devstack@n-api.service[72506]: X-User-Name: tempest-SnapshotDataIntegrityTests-1739495878-project-member Apr 12 02:10:11 user devstack@n-api.service[72506]: method: > Apr 12 02:10:11 user devstack@n-api.service[72506]: exception: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. Apr 12 02:10:11 user devstack@n-api.service[72506]: action_args: {'id': '101410fb-7682-46d8-baf1-18fcebf81cb8'} Apr 12 02:10:11 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. Apr 12 02:10:11 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:10:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Returning 404 to user: Instance 101410fb-7682-46d8-baf1-18fcebf81cb8 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:10:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4464df62-331d-424b-a559-d6911caaaaf1 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8" status: 404 len: 111 microversion: 2.1 time: 0.030789 Apr 12 02:10:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 261/772] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:10:11 2023] GET /compute/v2.1/servers/101410fb-7682-46d8-baf1-18fcebf81cb8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:10:12 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:12 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e046740c-1074-4aa2-b22e-7a29297767c3 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] Calling method '.version_select at 0x7fb202f3e950>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:10:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e046740c-1074-4aa2-b22e-7a29297767c3 tempest-SnapshotDataIntegrityTests-1739495878 tempest-SnapshotDataIntegrityTests-1739495878-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-SnapshotDataIntegrityTests-522022221" status: 202 len: 0 microversion: 2.1 time: 0.023700 Apr 12 02:10:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 259/773] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:10:12 2023] DELETE /compute/v2.1/os-keypairs/tempest-SnapshotDataIntegrityTests-522022221 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:19 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:10:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-012bcc9f-f3f2-416f-9413-3ccaa7a3a3f7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 200 len: 1548 microversion: 2.1 time: 0.409582 Apr 12 02:10:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 254/774] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:10:19 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 1548 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=19,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:10:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c927023f-f3ad-41c0-8b36-927d8c0f3ae0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1546 microversion: 2.1 time: 0.363520 Apr 12 02:10:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 262/775] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:10:37 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1546 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=22,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:10:49 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:10:49 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7c910aeb-871d-4e1c-bc06-063699b70ca5 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 200 len: 1544 microversion: 2.1 time: 0.277431 Apr 12 02:10:49 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 260/776] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:10:49 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 1544 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:19 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:11:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-730ebb23-c306-4d80-8a85-cc6925879080 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 200 len: 1548 microversion: 2.1 time: 0.151247 Apr 12 02:11:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 255/777] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:11:19 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 1548 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:19 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:11:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-faf77b52-e62e-4326-bb00-417f55aed79a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 204 len: 0 microversion: 2.1 time: 0.123296 Apr 12 02:11:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 263/778] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:11:19 2023] DELETE /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:20 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:11:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4d165ab4-9ff7-45ee-9691-bfb997df591a tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 200 len: 1554 microversion: 2.1 time: 0.188317 Apr 12 02:11:20 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 261/779] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:11:19 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 1554 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:11:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "name": "network-vif-unplugged", "status": "completed", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:11:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] Creating event network-vif-unplugged:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:11:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-766b63d0-8738-45be-8594-3420ee8005f2 req-b661c4e6-325c-4261-9d01-295cb1386e19 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.120667 Apr 12 02:11:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 256/780] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:11:20 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:11:22 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "name": "network-vif-plugged", "status": "completed", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}, {"server_uuid": "2f23b57e-3bf8-442a-8072-2ae145f9104a", "name": "network-vif-deleted", "tag": "1535b023-7f14-4e27-9691-f6728b943cfe"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:11:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Creating event network-vif-plugged:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:11:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] Creating event network-vif-deleted:1535b023-7f14-4e27-9691-f6728b943cfe for instance 2f23b57e-3bf8-442a-8072-2ae145f9104a on user Apr 12 02:11:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-dfcd14b1-11a5-452c-a2cd-08fa408b766c req-5cae1fa6-4b26-4c91-affd-3130bc5c1f8b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.031955 Apr 12 02:11:22 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 264/781] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:11:22 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:11:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1837199c-cc85-46ca-b507-ecc8da28a605 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1546 microversion: 2.1 time: 0.266006 Apr 12 02:11:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 262/782] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:11:37 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1546 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-dc20e2bc-f16a-4609-9eec-78a5a411ed7a tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action" status: 200 len: 28525 microversion: 2.1 time: 0.258710 Apr 12 02:11:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 257/783] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:11:38 2023] POST /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71/action => generated 28525 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:38 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:11:38 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6eccfc9d-4578-499a-98a0-4a23dc77a456 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 200 len: 1538 microversion: 2.1 time: 0.164900 Apr 12 02:11:38 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 265/784] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:11:38 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 1538 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:49 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-100b1ee3-4f86-48a0-b6e6-51fdbbf0d47e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-interface" status: 200 len: 282 microversion: 2.1 time: 0.103482 Apr 12 02:11:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 263/785] 10.0.0.210 () {62 vars in 1352 bytes} [Wed Apr 12 02:11:49 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-interface => generated 282 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] Creating event network-changed:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:11:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a39ab86c-bcd4-4324-b3a6-1461144c3038 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/ips" status: 200 len: 107 microversion: 2.1 time: 0.038164 Apr 12 02:11:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 266/786] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:11:50 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/ips => generated 107 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:11:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-b3c4395d-6078-41ef-aaa6-fffd3958e94f req-8d4f0460-31b0-4eb8-a360-74762c4c6900 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093610 Apr 12 02:11:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 258/787] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:11:50 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "da4c81dd-019a-4266-b972-a31e79e201c9"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:11:52 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:11:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a532d06f-ac1c-4358-ab56-d174644f8141 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "POST /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.281836 Apr 12 02:11:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 264/788] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:11:52 2023] POST /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments => generated 194 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:00 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lazy-loading 'services' on Instance uuid afccc0c5-9184-4a0c-a11e-50618b6e528d {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:12:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b5e21ba7-7a84-4125-bc62-5f9c4d00a7a9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments/da4c81dd-019a-4266-b972-a31e79e201c9" status: 202 len: 0 microversion: 2.1 time: 0.160965 Apr 12 02:12:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 267/789] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:12:00 2023] DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments/da4c81dd-019a-4266-b972-a31e79e201c9 => generated 0 bytes in 162 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6f4c038e-564f-4159-a79d-321ae24ba6f3 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056894 Apr 12 02:12:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 259/790] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:12:00 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=15,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:12:10 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a HTTP/1.1 Apr 12 02:12:10 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:12:10 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:12:10 user devstack@n-api.service[72507]: Connection: close Apr 12 02:12:10 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:12:10 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:12:10 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:12:10 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:12:10 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:12:10 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:12:10 user devstack@n-api.service[72507]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:12:10 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:12:10 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:12:10 user devstack@n-api.service[72507]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:12:10 user devstack@n-api.service[72507]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:12:10 user devstack@n-api.service[72507]: method: > Apr 12 02:12:10 user devstack@n-api.service[72507]: exception: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:12:10 user devstack@n-api.service[72507]: action_args: {'id': '2f23b57e-3bf8-442a-8072-2ae145f9104a'} Apr 12 02:12:10 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:12:10 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:12:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:12:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-9ce0bdb8-228a-49e9-9d00-284b87634613 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 404 len: 111 microversion: 2.1 time: 0.049007 Apr 12 02:12:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 265/791] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:12:10 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=13,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:12 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:12:12 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-1a5b716d-edaf-4162-8f60-c3b8ead6830f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 204 len: 0 microversion: 2.1 time: 0.111107 Apr 12 02:12:12 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 268/792] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:12:12 2023] DELETE /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:12 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:12:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2daa60b9-8a52-404d-b511-67fbeee81926 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 200 len: 1399 microversion: 2.1 time: 0.244148 Apr 12 02:12:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 260/793] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:12:12 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 1399 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:12:12 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "name": "network-vif-unplugged", "status": "completed", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] Creating event network-vif-unplugged:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:12:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-be6413a8-d2a1-434b-b55f-954c38771383 req-9350aa9e-66fd-4455-88bd-a7d514ac082e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.083386 Apr 12 02:12:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 266/794] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:12 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "name": "network-vif-deleted", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] Creating event network-vif-deleted:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:12:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-fedd6d3b-cf01-4cdb-af2a-ad275fc2696a req-9ab8317d-ed27-44b5-98fc-b05f91b5f024 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028941 Apr 12 02:12:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 269/795] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c", "name": "network-vif-plugged", "status": "completed", "tag": "e7679d13-0bc6-48e2-83ad-5a1c9fe8445c"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] Creating event network-vif-plugged:e7679d13-0bc6-48e2-83ad-5a1c9fe8445c for instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c on user Apr 12 02:12:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-27f1cfba-041e-424b-8d06-78afbc7fc1d9 req-35be83da-d444-48ad-b0d1-cd42f7ea5bc8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035976 Apr 12 02:12:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 261/796] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:38 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:12:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-eb1f4cd0-ae4d-4fe8-a627-648fb3b8415e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 204 len: 0 microversion: 2.1 time: 0.124766 Apr 12 02:12:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 267/797] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:12:38 2023] DELETE /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:12:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-unplugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] Creating event network-vif-unplugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:12:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-186b192c-0d29-464e-85df-3e2f4184a9de req-033ea1dc-dc79-4a82-a3c8-de27f14a42f5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039814 Apr 12 02:12:39 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 262/798] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:39 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:39 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:12:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-808cdd1f-6b82-4731-bce8-807b7dda93c4 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 200 len: 1552 microversion: 2.1 time: 0.594475 Apr 12 02:12:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 270/799] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:12:38 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 1552 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:12:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-plugged", "status": "completed", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}, {"server_uuid": "e4ae774a-0f87-4d51-a997-c8f9b228fe71", "name": "network-vif-deleted", "tag": "ce1a5774-7c99-4d64-b16b-2dab8491ff6d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Creating event network-vif-plugged:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:12:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] Creating event network-vif-deleted:ce1a5774-7c99-4d64-b16b-2dab8491ff6d for instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 on user Apr 12 02:12:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-d274193b-e619-40fe-8d18-1d9ff10c84ea req-8760c34a-6fb8-4b19-b9f3-52df15e4e2bf service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.031240 Apr 12 02:12:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 268/800] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:41 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-43a2c90d-4395-46de-8503-90557936e808 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050503 Apr 12 02:12:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 263/801] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:12:50 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:12:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-81572d32-2746-4f43-a618-4655ffd315f6 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 204 len: 0 microversion: 2.1 time: 0.119218 Apr 12 02:12:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 271/802] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:12:50 2023] DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:50 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '189b7aba48014bedacfa021cc3cad5e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2acf278c981e481fbe43e415c76aa6e7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:12:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d8ae0e87-58e8-41f0-9c3f-7a0060cc2fd9 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 200 len: 1668 microversion: 2.1 time: 0.201863 Apr 12 02:12:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 269/803] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:12:50 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 1668 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:12:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "name": "network-vif-unplugged", "status": "completed", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] Creating event network-vif-unplugged:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:12:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f19c3531-aa70-4e32-b96b-6c2aeeb2b091 req-ba14c33c-fd77-49c8-b436-35cb3d8e6439 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.027941 Apr 12 02:12:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 264/804] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:51 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "name": "network-vif-deleted", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] Creating event network-vif-deleted:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:12:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4522b442-d693-4129-892b-a8e31e09f138 req-330336f9-1e62-4721-ac8e-750b2ef27d0f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029751 Apr 12 02:12:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 272/805] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:52 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:12:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "afccc0c5-9184-4a0c-a11e-50618b6e528d", "name": "network-vif-plugged", "status": "completed", "tag": "50c67357-42ce-4826-b568-696829fac338"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:12:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:12:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:12:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:12:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] Creating event network-vif-plugged:50c67357-42ce-4826-b568-696829fac338 for instance afccc0c5-9184-4a0c-a11e-50618b6e528d on user Apr 12 02:12:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b62bbda0-9f56-4188-8c50-f40232b751d2 req-f177fd07-73d4-4191-8b7e-6f45d0fc27c4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034214 Apr 12 02:12:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 270/806] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:12:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:13:02 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c HTTP/1.1 Apr 12 02:13:02 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:02 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:02 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:02 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:02 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:02 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:02 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:02 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:02 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:02 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:02 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:02 user devstack@n-api.service[72505]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:13:02 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:02 user devstack@n-api.service[72505]: method: > Apr 12 02:13:02 user devstack@n-api.service[72505]: exception: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:02 user devstack@n-api.service[72505]: action_args: {'id': 'fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c'} Apr 12 02:13:02 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:02 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-16fba4cb-4f8f-424b-976e-a95c2e7564de tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 404 len: 111 microversion: 2.1 time: 0.038296 Apr 12 02:13:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 265/807] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:02 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:13:02 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a HTTP/1.1 Apr 12 02:13:02 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:02 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:02 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:02 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:02 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:02 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:02 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:02 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:02 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:02 user devstack@n-api.service[72506]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:02 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:02 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:02 user devstack@n-api.service[72506]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:13:02 user devstack@n-api.service[72506]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:02 user devstack@n-api.service[72506]: method: > Apr 12 02:13:02 user devstack@n-api.service[72506]: exception: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:02 user devstack@n-api.service[72506]: action_args: {'id': '2f23b57e-3bf8-442a-8072-2ae145f9104a'} Apr 12 02:13:02 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:02 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a745adf2-bdd1-4fac-9427-d46d2a7e02e5 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 404 len: 111 microversion: 2.1 time: 0.034943 Apr 12 02:13:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 273/808] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:02 2023] DELETE /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 2f23b57e-3bf8-442a-8072-2ae145f9104a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:13:03 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a HTTP/1.1 Apr 12 02:13:03 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:03 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:03 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:03 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:03 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:03 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:03 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:03 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72507]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72507]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:13:03 user devstack@n-api.service[72507]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72507]: method: > Apr 12 02:13:03 user devstack@n-api.service[72507]: exception: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:03 user devstack@n-api.service[72507]: action_args: {'id': '2f23b57e-3bf8-442a-8072-2ae145f9104a'} Apr 12 02:13:03 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. Apr 12 02:13:03 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 2f23b57e-3bf8-442a-8072-2ae145f9104a could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7a6320a4-8b15-4b6b-96dd-5f88699d8f02 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a" status: 404 len: 111 microversion: 2.1 time: 0.032264 Apr 12 02:13:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 271/809] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:02 2023] GET /compute/v2.1/servers/2f23b57e-3bf8-442a-8072-2ae145f9104a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:13:03 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c HTTP/1.1 Apr 12 02:13:03 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:03 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:03 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:03 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:03 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:03 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:03 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:03 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72505]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:13:03 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72505]: method: > Apr 12 02:13:03 user devstack@n-api.service[72505]: exception: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72505]: action_args: {'id': 'fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c'} Apr 12 02:13:03 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-72963da1-eec5-41ce-8e9d-917f8fb8c5e4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 404 len: 111 microversion: 2.1 time: 0.031400 Apr 12 02:13:03 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 266/810] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:03 2023] DELETE /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:03 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:13:03 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c HTTP/1.1 Apr 12 02:13:03 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:03 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:03 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:03 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:03 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:03 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:03 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:03 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:13:03 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:13:03 user devstack@n-api.service[72506]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:03 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:03 user devstack@n-api.service[72506]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:13:03 user devstack@n-api.service[72506]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:13:03 user devstack@n-api.service[72506]: method: > Apr 12 02:13:03 user devstack@n-api.service[72506]: exception: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72506]: action_args: {'id': 'fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c'} Apr 12 02:13:03 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. Apr 12 02:13:03 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:03 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:03 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7aad3377-cc40-42c5-a0ef-be684bc5586f tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c" status: 404 len: 111 microversion: 2.1 time: 0.028795 Apr 12 02:13:03 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 274/811] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:03 2023] GET /compute/v2.1/servers/fd0e54b9-aff2-4d86-ba56-b5eb7654ac9c => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fefad41c-d5cd-4926-8a15-9e62db3739bb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: .version_select at 0x7fb202f3dcf0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1281564430", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCZLzl+dJPPiQ1Q3KddtYvgtfp4Cw3N4yjPTn4jGAIGgcnNkNcoYvsYhuJ1X9VE6SsNAdPUgVuioRnH9/wkBTudNIYOOGxrInu47Q267Lh/dzVGncnNeYWswwovQJnMLew=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-fefad41c-d5cd-4926-8a15-9e62db3739bb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:13:03 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-fefad41c-d5cd-4926-8a15-9e62db3739bb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:13:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-fefad41c-d5cd-4926-8a15-9e62db3739bb tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.050517 Apr 12 02:13:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 272/812] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:13:03 2023] POST /compute/v2.1/os-keypairs => generated 400 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1377116913", "imageRef": "", "flavorRef": "42", "key_name": "tempest-TestVolumeBootPattern-1281564430", "security_groups": [{"name": "tempest-secgroup-smoke-150038052"}], "block_device_mapping_v2": [{"uuid": "f0dc4ffd-98fa-4a8c-8843-73706a802dc5", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:07 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:13:08 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:13:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:13:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='f0dc4ffd-98fa-4a8c-8843-73706a802dc5',volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:13:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0810e3bb-281f-49b6-878e-3a6f91db97a4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 0.851465 Apr 12 02:13:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 267/813] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:13:07 2023] POST /compute/v2.1/servers => generated 403 bytes in 853 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:13:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b1048d35-9b74-49a8-9f94-aacedfaa8506 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b1048d35-9b74-49a8-9f94-aacedfaa8506 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b1048d35-9b74-49a8-9f94-aacedfaa8506 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:13:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b1048d35-9b74-49a8-9f94-aacedfaa8506 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 200 len: 1090 microversion: 2.1 time: 0.065289 Apr 12 02:13:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 275/814] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:08 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 1090 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=27,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] Creating event network-changed:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:13:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-6a9c45d6-9442-4ebd-a60b-6a9e47bc184a req-cbc1f9f5-2c32-46fe-b401-7af61610a6e1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037706 Apr 12 02:13:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 273/815] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:13:10 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "name": "network-vif-plugged", "status": "completed", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] Creating event network-vif-plugged:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:13:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-5ebe2827-e850-4f6b-8cd9-edde15aa6029 req-35ce2786-85a5-4668-adf9-cb1525357e96 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031418 Apr 12 02:13:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 268/816] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:13:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "name": "network-vif-plugged", "status": "completed", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] Creating event network-vif-plugged:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:13:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-0c05d185-1d09-409d-9b7b-51e768664083 req-a93c7422-7055-4c73-ae32-5ddc8df3069b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036428 Apr 12 02:13:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 276/817] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:13:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:13:29 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 HTTP/1.1 Apr 12 02:13:29 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:29 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:29 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:29 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:29 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:29 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:29 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:29 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72507]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72507]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:13:29 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72507]: method: > Apr 12 02:13:29 user devstack@n-api.service[72507]: exception: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72507]: action_args: {'id': 'e4ae774a-0f87-4d51-a997-c8f9b228fe71'} Apr 12 02:13:29 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-fa426ffa-3841-4afc-9492-69b3e0cfc581 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 404 len: 111 microversion: 2.1 time: 0.037302 Apr 12 02:13:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 274/818] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:29 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:13:29 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 HTTP/1.1 Apr 12 02:13:29 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:29 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:29 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:29 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:29 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:29 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:29 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:29 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72505]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72505]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:13:29 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72505]: method: > Apr 12 02:13:29 user devstack@n-api.service[72505]: exception: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72505]: action_args: {'id': 'e4ae774a-0f87-4d51-a997-c8f9b228fe71'} Apr 12 02:13:29 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7daedb4a-0477-43a6-918e-c43c9ce99fca tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 404 len: 111 microversion: 2.1 time: 0.037241 Apr 12 02:13:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 269/819] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:29 2023] DELETE /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: e4ae774a-0f87-4d51-a997-c8f9b228fe71] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:13:29 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 HTTP/1.1 Apr 12 02:13:29 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:29 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:29 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:29 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:29 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:29 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:29 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:29 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:13:29 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:13:29 user devstack@n-api.service[72506]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:29 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:29 user devstack@n-api.service[72506]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:13:29 user devstack@n-api.service[72506]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:13:29 user devstack@n-api.service[72506]: method: > Apr 12 02:13:29 user devstack@n-api.service[72506]: exception: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72506]: action_args: {'id': 'e4ae774a-0f87-4d51-a997-c8f9b228fe71'} Apr 12 02:13:29 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. Apr 12 02:13:29 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance e4ae774a-0f87-4d51-a997-c8f9b228fe71 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-abe67390-555c-49f2-a1d2-11beac771361 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71" status: 404 len: 111 microversion: 2.1 time: 0.032559 Apr 12 02:13:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 277/820] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:29 2023] GET /compute/v2.1/servers/e4ae774a-0f87-4d51-a997-c8f9b228fe71 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:13:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d9691597-12d3-48ae-9280-7072e09c6ab0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 204 len: 0 microversion: 2.1 time: 0.117963 Apr 12 02:13:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 275/821] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:29 2023] DELETE /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:30 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "name": "network-vif-unplugged", "status": "completed", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:30 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] Creating event network-vif-unplugged:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:13:30 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c0cdcbf5-c643-41fa-8cfa-f27018b51894 req-4fc107e5-4329-486d-a807-d0093e2d2ee0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054945 Apr 12 02:13:30 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 278/822] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:13:30 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:30 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e8e8b9f0307b4d3fb54d89962798026b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '117589cba4a54dd581f4ec5c26b4cd2a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:13:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c2777e62-5c3d-4fc7-9459-c53d3cc5bdbf tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 200 len: 1544 microversion: 2.1 time: 0.578122 Apr 12 02:13:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 270/823] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:29 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 1544 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:13:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "name": "network-vif-plugged", "status": "completed", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}, {"server_uuid": "1a3c868f-3a4c-430e-8009-7a855e832f82", "name": "network-vif-deleted", "tag": "915517f3-521b-4d3a-a7c0-841b64ac50a8"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Creating event network-vif-plugged:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:13:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] Creating event network-vif-deleted:915517f3-521b-4d3a-a7c0-841b64ac50a8 for instance 1a3c868f-3a4c-430e-8009-7a855e832f82 on user Apr 12 02:13:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-32a2cf00-b3db-45be-8b15-17446e7f4efe req-e1e171d6-f4fd-422b-9449-2ae866008a65 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.030249 Apr 12 02:13:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 276/824] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:13:32 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:41 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d HTTP/1.1 Apr 12 02:13:41 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:41 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:41 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:41 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:41 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:41 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:41 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:41 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:41 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:41 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:41 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:41 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:41 user devstack@n-api.service[72506]: method: > Apr 12 02:13:41 user devstack@n-api.service[72506]: exception: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:41 user devstack@n-api.service[72506]: action_args: {'id': 'afccc0c5-9184-4a0c-a11e-50618b6e528d'} Apr 12 02:13:41 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:41 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-881f230d-0ab9-454c-ac5e-84706711dba1 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 404 len: 111 microversion: 2.1 time: 0.036151 Apr 12 02:13:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 279/825] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:40 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87971d26-8ae7-430e-99e9-c659fb584c9c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '.version_select at 0x7fb203092200>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-87971d26-8ae7-430e-99e9-c659fb584c9c tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-725616642" status: 202 len: 0 microversion: 2.1 time: 0.024151 Apr 12 02:13:41 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 271/826] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:13:41 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-725616642 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:43 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d HTTP/1.1 Apr 12 02:13:43 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:43 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:43 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:43 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:43 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:43 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:43 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:43 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:43 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:43 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:43 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:43 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:43 user devstack@n-api.service[72507]: method: > Apr 12 02:13:43 user devstack@n-api.service[72507]: exception: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72507]: action_args: {'id': 'afccc0c5-9184-4a0c-a11e-50618b6e528d'} Apr 12 02:13:43 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a737a578-f6cc-46dd-ac46-7f783eda56ee tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 404 len: 111 microversion: 2.1 time: 0.036862 Apr 12 02:13:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 277/827] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:43 2023] DELETE /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: afccc0c5-9184-4a0c-a11e-50618b6e528d] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:43 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d HTTP/1.1 Apr 12 02:13:43 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:43 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:43 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:43 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:43 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:43 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:43 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:43 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:43 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:43 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:43 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:43 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:43 user devstack@n-api.service[72506]: method: > Apr 12 02:13:43 user devstack@n-api.service[72506]: exception: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72506]: action_args: {'id': 'afccc0c5-9184-4a0c-a11e-50618b6e528d'} Apr 12 02:13:43 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. Apr 12 02:13:43 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance afccc0c5-9184-4a0c-a11e-50618b6e528d could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-91d9f269-892c-4a3c-8bf1-d3ccfc71973e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d" status: 404 len: 111 microversion: 2.1 time: 0.026974 Apr 12 02:13:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 280/828] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:43 2023] GET /compute/v2.1/servers/afccc0c5-9184-4a0c-a11e-50618b6e528d => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:46 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 HTTP/1.1 Apr 12 02:13:46 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:46 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:46 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:46 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:46 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:46 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:46 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:46 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72505]: method: > Apr 12 02:13:46 user devstack@n-api.service[72505]: exception: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: action_args: {'id': '34a07ab6-379e-48d9-98ae-c32981534331'} Apr 12 02:13:46 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-87029ef5-803d-4090-9f0d-9b6656964c89 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 404 len: 111 microversion: 2.1 time: 0.031907 Apr 12 02:13:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 272/829] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:46 2023] DELETE /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 34a07ab6-379e-48d9-98ae-c32981534331] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:46 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 HTTP/1.1 Apr 12 02:13:46 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:46 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:46 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:46 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:46 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:46 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:46 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:46 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:46 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72507]: method: > Apr 12 02:13:46 user devstack@n-api.service[72507]: exception: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72507]: action_args: {'id': '34a07ab6-379e-48d9-98ae-c32981534331'} Apr 12 02:13:46 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 34a07ab6-379e-48d9-98ae-c32981534331 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7f8ed009-e13e-4530-9b80-47146e9ce5ff tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331" status: 404 len: 111 microversion: 2.1 time: 0.026570 Apr 12 02:13:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 278/830] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:46 2023] GET /compute/v2.1/servers/34a07ab6-379e-48d9-98ae-c32981534331 => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:46 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 HTTP/1.1 Apr 12 02:13:46 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:46 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:46 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:46 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:46 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:46 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:46 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:46 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:46 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72506]: method: > Apr 12 02:13:46 user devstack@n-api.service[72506]: exception: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72506]: action_args: {'id': '0fa3c7e6-a1a9-490e-ae02-8b1359736485'} Apr 12 02:13:46 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-a5116490-0efb-45d7-a28d-e87cab5c4470 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 404 len: 111 microversion: 2.1 time: 0.028031 Apr 12 02:13:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 281/831] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:46 2023] DELETE /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 0fa3c7e6-a1a9-490e-ae02-8b1359736485] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:46 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 HTTP/1.1 Apr 12 02:13:46 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:46 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:46 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:46 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:46 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:46 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:46 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:46 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:46 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:46 user devstack@n-api.service[72505]: method: > Apr 12 02:13:46 user devstack@n-api.service[72505]: exception: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: action_args: {'id': '0fa3c7e6-a1a9-490e-ae02-8b1359736485'} Apr 12 02:13:46 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. Apr 12 02:13:46 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 0fa3c7e6-a1a9-490e-ae02-8b1359736485 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b319272c-18f9-4dfa-8930-5a620b3e956e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485" status: 404 len: 111 microversion: 2.1 time: 0.029167 Apr 12 02:13:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 273/832] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:46 2023] GET /compute/v2.1/servers/0fa3c7e6-a1a9-490e-ae02-8b1359736485 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:48 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd HTTP/1.1 Apr 12 02:13:48 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:48 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:48 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:48 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:48 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:48 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:48 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:48 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:48 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:48 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:48 user devstack@n-api.service[72507]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:48 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:48 user devstack@n-api.service[72507]: method: > Apr 12 02:13:48 user devstack@n-api.service[72507]: exception: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72507]: action_args: {'id': '92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd'} Apr 12 02:13:48 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7875a581-c354-4560-afdf-8636fc20418e tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 404 len: 111 microversion: 2.1 time: 0.026297 Apr 12 02:13:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 279/833] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:13:48 2023] DELETE /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] [instance: 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] HTTP exception thrown: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Request method failure captured: Apr 12 02:13:48 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd HTTP/1.1 Apr 12 02:13:48 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:48 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:48 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:48 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:13:48 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:48 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:48 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:48 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhCHzVvgHMWYAEh8YfBO6EZ_0Y6KiFaSkCnbjG62Wity0h8X5Q2mL5tm4gJRCsBD0THfzUoqwKBGt8pYVgcgCsSZx32E_NaXXnHKhI-VJOXyOiIvxb5Voaiq4z03stCBITADd3O9Wc1fWQ4A8K4HjwIVU6_jNzzGuouzMJfKA3TlDcM Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Project-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/2acf278c981e481fbe43e415c76aa6e7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Tenant-Id: 2acf278c981e481fbe43e415c76aa6e7 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1849748479 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:48 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:48 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:48 user devstack@n-api.service[72506]: X-User-Id: 189b7aba48014bedacfa021cc3cad5e4 Apr 12 02:13:48 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeNegativeTest-1849748479-project-member Apr 12 02:13:48 user devstack@n-api.service[72506]: method: > Apr 12 02:13:48 user devstack@n-api.service[72506]: exception: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72506]: action_args: {'id': '92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd'} Apr 12 02:13:48 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. Apr 12 02:13:48 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:48 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] Returning 404 to user: Instance 92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:48 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4bf5e0e8-2021-41c2-ab52-be96ebc32e33 tempest-AttachVolumeNegativeTest-1849748479 tempest-AttachVolumeNegativeTest-1849748479-project-member] 10.0.0.210 "GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd" status: 404 len: 111 microversion: 2.1 time: 0.027797 Apr 12 02:13:48 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 282/834] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:48 2023] GET /compute/v2.1/servers/92e8b4c7-58a2-4deb-b9a5-5de7d68c82fd => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-35e25f6a-cd74-49da-a3f2-62f34766f28f tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Action: 'create', calling method: .version_select at 0x7fb203091bd0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-609722013", "metadata": {"Type": "work"}}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-35e25f6a-cd74-49da-a3f2-62f34766f28f tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72505]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Length: 121 Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: Apr 12 02:13:52 user devstack@n-api.service[72505]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-609722013", "metadata": {"Type": "work"}}} Apr 12 02:13:52 user devstack@n-api.service[72505]: method: Apr 12 02:13:52 user devstack@n-api.service[72505]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72505]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-609722013', 'metadata': {'Type': 'work'}}}} Apr 12 02:13:52 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 314, in validate Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise error Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi On instance['volume']: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi '#$%' Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-35e25f6a-cd74-49da-a3f2-62f34766f28f tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-35e25f6a-cd74-49da-a3f2-62f34766f28f tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.184322 Apr 12 02:13:52 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 274/835] 10.0.0.210 () {64 vars in 1233 bytes} [Wed Apr 12 02:13:52 2023] POST /compute/v2.1/os-volumes => generated 127 bytes in 186 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2b6448be-88a5-47ee-a0c8-0db52c9d5680 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Action: 'create', calling method: .version_select at 0x7fb202f0a200>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-393636966", "metadata": {"Type": "work"}}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2b6448be-88a5-47ee-a0c8-0db52c9d5680 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72507]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72507]: Content-Length: 119 Apr 12 02:13:52 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72507]: Apr 12 02:13:52 user devstack@n-api.service[72507]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-393636966", "metadata": {"Type": "work"}}} Apr 12 02:13:52 user devstack@n-api.service[72507]: method: Apr 12 02:13:52 user devstack@n-api.service[72507]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Apr 12 02:13:52 user devstack@n-api.service[72507]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-393636966', 'metadata': {'Type': 'work'}}}} Apr 12 02:13:52 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 314, in validate Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise error Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi On instance['volume']: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi '0' Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-2b6448be-88a5-47ee-a0c8-0db52c9d5680 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-2b6448be-88a5-47ee-a0c8-0db52c9d5680 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009312 Apr 12 02:13:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 280/836] 10.0.0.210 () {64 vars in 1233 bytes} [Wed Apr 12 02:13:52 2023] POST /compute/v2.1/os-volumes => generated 125 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4811173c-6a04-443d-9a53-8e5d4b9debd5 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Action: 'create', calling method: .version_select at 0x7fb202e44310>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1839736008", "metadata": {"Type": "work"}}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:13:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4811173c-6a04-443d-9a53-8e5d4b9debd5 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72506]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72506]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72506]: Content-Length: 119 Apr 12 02:13:52 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72506]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72506]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72506]: Apr 12 02:13:52 user devstack@n-api.service[72506]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1839736008", "metadata": {"Type": "work"}}} Apr 12 02:13:52 user devstack@n-api.service[72506]: method: Apr 12 02:13:52 user devstack@n-api.service[72506]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72506]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1839736008', 'metadata': {'Type': 'work'}}}} Apr 12 02:13:52 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 314, in validate Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise error Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi On instance['volume']: Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi '' Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Apr 12 02:13:52 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-4811173c-6a04-443d-9a53-8e5d4b9debd5 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-4811173c-6a04-443d-9a53-8e5d4b9debd5 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010634 Apr 12 02:13:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 283/837] 10.0.0.210 () {64 vars in 1233 bytes} [Wed Apr 12 02:13:52 2023] POST /compute/v2.1/os-volumes => generated 121 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-40e82175-718a-4674-8433-de3e020e7e40 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Calling method '.version_select at 0x7fb203091fc0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-40e82175-718a-4674-8433-de3e020e7e40 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] HTTP exception thrown: Volume tempest-invalid-1595570046 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-40e82175-718a-4674-8433-de3e020e7e40 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1595570046 HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: method: Apr 12 02:13:52 user devstack@n-api.service[72505]: exception: Volume tempest-invalid-1595570046 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: action_args: {'id': 'tempest-invalid-1595570046'} Apr 12 02:13:52 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1595570046 could not be found. (HTTP 404) (Request-ID: req-766f1c31-0210-4609-84f5-768a83545c44) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1595570046 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1595570046 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1595570046 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-40e82175-718a-4674-8433-de3e020e7e40 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 404 to user: Volume tempest-invalid-1595570046 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-40e82175-718a-4674-8433-de3e020e7e40 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1595570046" status: 404 len: 99 microversion: 2.1 time: 0.036026 Apr 12 02:13:52 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 275/838] 10.0.0.210 () {62 vars in 1295 bytes} [Wed Apr 12 02:13:52 2023] DELETE /compute/v2.1/os-volumes/tempest-invalid-1595570046 => generated 99 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-97113188-6e52-4a86-ac46-feabac54ca26 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003937 Apr 12 02:13:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 281/839] 10.0.0.210 () {62 vars in 1217 bytes} [Wed Apr 12 02:13:52 2023] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cba21a1d-a2af-4150-9207-ad160390b4f3 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003706 Apr 12 02:13:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 284/840] 10.0.0.210 () {62 vars in 1214 bytes} [Wed Apr 12 02:13:52 2023] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f8078406-3c2e-4377-981a-60bf9532c881 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Calling method '.version_select at 0x7fb2030915a0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-f8078406-3c2e-4377-981a-60bf9532c881 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] HTTP exception thrown: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f8078406-3c2e-4377-981a-60bf9532c881 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/os-volumes/ac6cd925-3df1-4974-96f1-19428071a81f HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72505]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:13:52 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72505]: method: Apr 12 02:13:52 user devstack@n-api.service[72505]: exception: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: action_args: {'id': 'ac6cd925-3df1-4974-96f1-19428071a81f'} Apr 12 02:13:52 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. (HTTP 404) (Request-ID: req-8a41b038-6c96-4da9-814b-79b76bacb2b4) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. Apr 12 02:13:52 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f8078406-3c2e-4377-981a-60bf9532c881 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 404 to user: Volume ac6cd925-3df1-4974-96f1-19428071a81f could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f8078406-3c2e-4377-981a-60bf9532c881 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/ac6cd925-3df1-4974-96f1-19428071a81f" status: 404 len: 109 microversion: 2.1 time: 0.035473 Apr 12 02:13:52 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 276/841] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:13:52 2023] DELETE /compute/v2.1/os-volumes/ac6cd925-3df1-4974-96f1-19428071a81f => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b3fb770d-8ab8-4fb4-9435-ddd04b386271 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Calling method '.version_select at 0x7fb20301f370>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-b3fb770d-8ab8-4fb4-9435-ddd04b386271 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] HTTP exception thrown: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b3fb770d-8ab8-4fb4-9435-ddd04b386271 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Request method failure captured: Apr 12 02:13:52 user devstack@n-api.service[72507]: request: GET /compute/v2.1/os-volumes/b2f9e560-9e08-4a94-acc4-085afac0ff93 HTTP/1.1 Apr 12 02:13:52 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:13:52 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:13:52 user devstack@n-api.service[72507]: Connection: close Apr 12 02:13:52 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:13:52 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:13:52 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:13:52 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:13:52 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhPgFfSkFAUbOaokYJNn7HbhmPCSBgjDOAqdkPg2NthJ09Fkt4QmnpvcjoiCu_G1kOrZl72kG3TwzU2_ugG7Ju38mSAYTM05_c54vKwxtOQrgH1FESVm6sGgK-7h7gou8KX7mkir5ZfceYf_1yU_b9yQxpNTwfapHjKNYULvu24IURA Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/164637f648bb4598a810454b9de32a71"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/164637f648bb4598a810454b9de32a71"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant-Id: 164637f648bb4598a810454b9de32a71 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesNegativeTest-1041677812 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Id: 8f9b8cb71c354d7cadf9a3ca00305745 Apr 12 02:13:52 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesNegativeTest-1041677812-project-member Apr 12 02:13:52 user devstack@n-api.service[72507]: method: Apr 12 02:13:52 user devstack@n-api.service[72507]: exception: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72507]: action_args: {'id': 'b2f9e560-9e08-4a94-acc4-085afac0ff93'} Apr 12 02:13:52 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. (HTTP 404) (Request-ID: req-31619cee-2f9e-48a4-bf42-b710fc36879d) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. Apr 12 02:13:52 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:13:52 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b3fb770d-8ab8-4fb4-9435-ddd04b386271 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] Returning 404 to user: Volume b2f9e560-9e08-4a94-acc4-085afac0ff93 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:13:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b3fb770d-8ab8-4fb4-9435-ddd04b386271 tempest-VolumesNegativeTest-1041677812 tempest-VolumesNegativeTest-1041677812-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/b2f9e560-9e08-4a94-acc4-085afac0ff93" status: 404 len: 109 microversion: 2.1 time: 0.036070 Apr 12 02:13:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 282/842] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:13:52 2023] GET /compute/v2.1/os-volumes/b2f9e560-9e08-4a94-acc4-085afac0ff93 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:13:58 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:13:58 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-dbbc7e39-1bc1-4883-92ae-0b8f2f4a2859 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 200 len: 1443 microversion: 2.1 time: 0.145529 Apr 12 02:13:58 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 285/843] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:13:58 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 1443 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=7,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: 1a3c868f-3a4c-430e-8009-7a855e832f82] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:14:20 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 HTTP/1.1 Apr 12 02:14:20 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:14:20 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:14:20 user devstack@n-api.service[72505]: Connection: close Apr 12 02:14:20 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:14:20 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:14:20 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:14:20 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:14:20 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:20 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:20 user devstack@n-api.service[72505]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:20 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:14:20 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:14:20 user devstack@n-api.service[72505]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:14:20 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:20 user devstack@n-api.service[72505]: method: > Apr 12 02:14:20 user devstack@n-api.service[72505]: exception: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. Apr 12 02:14:20 user devstack@n-api.service[72505]: action_args: {'id': '1a3c868f-3a4c-430e-8009-7a855e832f82'} Apr 12 02:14:20 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. Apr 12 02:14:20 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:14:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance 1a3c868f-3a4c-430e-8009-7a855e832f82 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:14:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d474934d-2752-4994-ac4d-2781d901d27e tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82" status: 404 len: 111 microversion: 2.1 time: 0.042382 Apr 12 02:14:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 277/844] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:14:20 2023] GET /compute/v2.1/servers/1a3c868f-3a4c-430e-8009-7a855e832f82 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:14:25 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 HTTP/1.1 Apr 12 02:14:25 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:14:25 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:14:25 user devstack@n-api.service[72507]: Connection: close Apr 12 02:14:25 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:14:25 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:14:25 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:14:25 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:14:25 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:25 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72507]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:25 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:14:25 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:14:25 user devstack@n-api.service[72507]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:14:25 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:25 user devstack@n-api.service[72507]: method: > Apr 12 02:14:25 user devstack@n-api.service[72507]: exception: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72507]: action_args: {'id': 'c712f2db-72cb-4136-a719-56057d82fff6'} Apr 12 02:14:25 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:14:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-22727def-b7fa-49dd-b181-0ebc2f24b3f0 tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 404 len: 111 microversion: 2.1 time: 0.041939 Apr 12 02:14:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 283/845] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:14:25 2023] DELETE /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] [instance: c712f2db-72cb-4136-a719-56057d82fff6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=10 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=14 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] HTTP exception thrown: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Request method failure captured: Apr 12 02:14:25 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 HTTP/1.1 Apr 12 02:14:25 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:14:25 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:14:25 user devstack@n-api.service[72506]: Connection: close Apr 12 02:14:25 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:14:25 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:14:25 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:14:25 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:14:25 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNg4yGx7r_JP3D98kOpOiiE0Zo8jV8PiKyx5RsYiDbv2zI9eNzoY2dcyDWsQ_PqEcLP8HixQSgPsljB7wHAkURoAYi_vN-oJkopqJBPIlpxRPvNRXgGCIRSLZ4Kcmnj9Tq8oPLaQsrLVUqCYJj4ZkK7-sLziw0dDhvPIFXIPsIYiLE1w Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Project-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Project-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/117589cba4a54dd581f4ec5c26b4cd2a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Tenant: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Tenant-Id: 117589cba4a54dd581f4ec5c26b4cd2a Apr 12 02:14:25 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-ServerActionsTestJSON-1409832124 Apr 12 02:14:25 user devstack@n-api.service[72506]: X-User: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:25 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:14:25 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:14:25 user devstack@n-api.service[72506]: X-User-Id: e8e8b9f0307b4d3fb54d89962798026b Apr 12 02:14:25 user devstack@n-api.service[72506]: X-User-Name: tempest-ServerActionsTestJSON-1409832124-project-member Apr 12 02:14:25 user devstack@n-api.service[72506]: method: > Apr 12 02:14:25 user devstack@n-api.service[72506]: exception: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72506]: action_args: {'id': 'c712f2db-72cb-4136-a719-56057d82fff6'} Apr 12 02:14:25 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. Apr 12 02:14:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Returning 404 to user: Instance c712f2db-72cb-4136-a719-56057d82fff6 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:14:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-77092d8b-17b7-45e1-9d6a-bb82250d8d0c tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6" status: 404 len: 111 microversion: 2.1 time: 0.033736 Apr 12 02:14:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 286/846] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:14:25 2023] GET /compute/v2.1/servers/c712f2db-72cb-4136-a719-56057d82fff6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:14:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3bb36c0b-7fcb-40af-9adf-a2df90bd47db tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] Calling method '.version_select at 0x7fb203043010>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3bb36c0b-7fcb-40af-9adf-a2df90bd47db tempest-ServerActionsTestJSON-1409832124 tempest-ServerActionsTestJSON-1409832124-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1931079911" status: 202 len: 0 microversion: 2.1 time: 0.025472 Apr 12 02:14:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 278/847] 10.0.0.210 () {62 vars in 1298 bytes} [Wed Apr 12 02:14:25 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1931079911 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e1708af7-d2d3-4bcf-b505-29dc30716327 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: .version_select at 0x7fb20301fc70>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1687627615", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC9jZCWKV6hxux1R3+C+7uefGfjpgPqW32T2YiNnkme9UOOl4uU3OGWAWXACSx7haDpHOtbD7cTM7r65siGiAgj09hq/+hBvt2qQXd3ZpwfsOyduBx4iSrwxw2ZDVRKj2g=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e1708af7-d2d3-4bcf-b505-29dc30716327 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for project 71b73a0cde174feeb85558cebb13d137. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:14:25 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-e1708af7-d2d3-4bcf-b505-29dc30716327 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for user 88570fbc28654337a032da5fa1f58814 and project 71b73a0cde174feeb85558cebb13d137. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:14:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e1708af7-d2d3-4bcf-b505-29dc30716327 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.050877 Apr 12 02:14:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 284/848] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:14:25 2023] POST /compute/v2.1/os-keypairs => generated 403 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1897008155", "imageRef": "844b59ea-88c3-4180-bac8-c8fed5694334", "flavorRef": "42", "key_name": "tempest-TestMinimumBasicScenario-1687627615", "networks": [{"uuid": "d62e0f89-6046-4a5f-9fda-0c742420c904"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:25 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] validate_networks() for [('d62e0f89-6046-4a5f-9fda-0c742420c904', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for project 71b73a0cde174feeb85558cebb13d137. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for user 88570fbc28654337a032da5fa1f58814 and project 71b73a0cde174feeb85558cebb13d137. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:14:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='844b59ea-88c3-4180-bac8-c8fed5694334',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:14:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cf4cd32b-7305-4d75-b521-0fe3c514503b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.611499 Apr 12 02:14:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 287/849] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:14:25 2023] POST /compute/v2.1/servers => generated 378 bytes in 616 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:14:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b87e9d95-ae08-4f45-9738-c9fe39f5a2db tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:26 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b87e9d95-ae08-4f45-9738-c9fe39f5a2db tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:26 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-b87e9d95-ae08-4f45-9738-c9fe39f5a2db tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:14:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b87e9d95-ae08-4f45-9738-c9fe39f5a2db tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1253 microversion: 2.1 time: 0.285837 Apr 12 02:14:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 279/850] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:14:26 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1253 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:14:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] Creating event network-changed:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:14:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-00278f3c-c35d-4faa-8f8b-a425696cc759 req-1ca04699-5503-4c60-99ed-76627e7361a7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036700 Apr 12 02:14:28 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 285/851] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:28 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:31 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:14:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ef93f2a2-fa0f-4e19-9991-32100663950f req-dcf9f809-7500-4a34-b920-710f4c3ca1f5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086100 Apr 12 02:14:31 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 288/852] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:31 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:33 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:14:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-630a040f-49ea-4c55-a723-db75935cf24e req-292fced9-9207-424f-9a2a-8e305a206319 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041444 Apr 12 02:14:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 280/853] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:33 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerBootFromVolumeStableRescueTest-server-1938777543", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "68f69bfd-e205-4f13-92e2-346de9c67dd4"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:38 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] validate_networks() for [('68f69bfd-e205-4f13-92e2-346de9c67dd4', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:14:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for user fe9217f4c1ff428086140b7fd07194eb and project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:14:39 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:14:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-069e3409-5e8e-4052-949a-2f423841c485 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.87 time: 0.936802 Apr 12 02:14:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 286/854] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:14:38 2023] POST /compute/v2.1/servers => generated 378 bytes in 938 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:14:39 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:14:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0966e846-159d-423b-b828-e0dd9a01289a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 200 len: 1395 microversion: 2.87 time: 0.095344 Apr 12 02:14:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 289/855] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:14:39 2023] GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 1395 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:14:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] Creating event network-changed:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:14:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-6142f0fb-bc53-45dd-b397-01abbbf1fd8b req-823b78ea-bce9-4950-986c-5d07a1e8f3b6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035672 Apr 12 02:14:41 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 281/856] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:41 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:14:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-fb07fa07-951c-4c99-b35e-bc795087788e req-53f81984-98dd-4420-a7b0-e26603923eed service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032753 Apr 12 02:14:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 287/857] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:43 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:45 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:14:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:14:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f1636ee9-0280-4f91-b965-d91f83953457 req-36afe8d0-3198-405f-a95d-bed963ee11be service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043486 Apr 12 02:14:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 290/858] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:14:45 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=13,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:14:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:14:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5a799dd1-1973-4bd2-aedc-b4953745234c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 200 len: 1443 microversion: 2.1 time: 0.284403 Apr 12 02:14:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 282/859] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:14:58 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 1443 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] Creating event network-changed:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:15:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-a6973f67-0c94-4859-b48e-fa84802ecf43 req-02673cc6-4e55-4f87-afb7-759a4652ccda service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035946 Apr 12 02:15:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 288/860] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:15:00 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:15:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0354072d-6fa3-4438-9487-cf2a302459c1 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 204 len: 0 microversion: 2.1 time: 0.289520 Apr 12 02:15:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 291/861] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:15:00 2023] DELETE /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:01 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-db097036-b2b3-42d3-8eb7-5e9f3ad79870 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 200 len: 1449 microversion: 2.1 time: 0.273526 Apr 12 02:15:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 283/862] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:15:00 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 1449 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:15:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "name": "network-vif-unplugged", "status": "completed", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] Creating event network-vif-unplugged:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:15:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-59ee3773-086c-49ea-ac1b-08002ec4bd9c req-34f4dd2a-e689-4ef3-b057-5e2be8a2129f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030955 Apr 12 02:15:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 289/863] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:15:01 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:15:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "name": "network-vif-deleted", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] Creating event network-vif-deleted:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:15:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-9af60cbf-bed8-40f0-a7e3-25ab5b27e543 req-ce1c1832-b0b3-4e02-87a6-3ef661b62f23 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037774 Apr 12 02:15:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 292/864] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:15:02 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:15:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc88ac73-1963-4055-bd73-7a51c0850528", "name": "network-vif-plugged", "status": "completed", "tag": "f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] Creating event network-vif-plugged:f6c96f05-2c3b-4ac9-aaea-efa1c67ab2da for instance fc88ac73-1963-4055-bd73-7a51c0850528 on user Apr 12 02:15:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-4ab4bc15-11ce-485b-abac-b394859f3faf req-a9ed227a-b49e-4658-9ac7-99bb3db189b5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031115 Apr 12 02:15:03 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 284/865] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:15:03 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:17 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3bce5e4c-52a4-4f1d-9cc3-005078499c5c tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1536 microversion: 2.1 time: 0.197065 Apr 12 02:15:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 290/866] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:15:16 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1536 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:29 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-54b79fec-2cf7-4ebc-85ab-17b17a897600 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 200 len: 1691 microversion: 2.87 time: 0.216598 Apr 12 02:15:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 293/867] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:15:29 2023] GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 1691 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:15:51 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 HTTP/1.1 Apr 12 02:15:51 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:15:51 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:15:51 user devstack@n-api.service[72505]: Connection: close Apr 12 02:15:51 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:15:51 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:15:51 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:15:51 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:15:51 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:15:51 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:15:51 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:15:51 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:15:51 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:15:51 user devstack@n-api.service[72505]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:15:51 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:15:51 user devstack@n-api.service[72505]: method: > Apr 12 02:15:51 user devstack@n-api.service[72505]: exception: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:15:51 user devstack@n-api.service[72505]: action_args: {'id': 'fc88ac73-1963-4055-bd73-7a51c0850528'} Apr 12 02:15:51 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:15:51 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:15:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:15:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-561feac1-4c4b-44b5-9581-173c5ee02dc9 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 404 len: 111 microversion: 2.1 time: 0.183305 Apr 12 02:15:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 285/868] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:15:51 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 111 bytes in 186 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1500776453", "imageRef": "", "flavorRef": "42", "key_name": "tempest-TestVolumeBootPattern-1281564430", "security_groups": [{"name": "tempest-secgroup-smoke-150038052"}], "block_device_mapping_v2": [{"uuid": "f0dc4ffd-98fa-4a8c-8843-73706a802dc5", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:15:51 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:15:52 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:15:52 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='f0dc4ffd-98fa-4a8c-8843-73706a802dc5',volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:15:52 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-00f01233-0207-4dec-99b6-8b872f8ebcae tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 0.769233 Apr 12 02:15:52 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 291/869] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:15:51 2023] POST /compute/v2.1/servers => generated 403 bytes in 771 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:15:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8b9d207f-954d-44f5-998f-4e91f77267b4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:15:52 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-8b9d207f-954d-44f5-998f-4e91f77267b4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:15:52 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:52 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:15:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-8b9d207f-954d-44f5-998f-4e91f77267b4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:15:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8b9d207f-954d-44f5-998f-4e91f77267b4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 200 len: 1090 microversion: 2.1 time: 0.113038 Apr 12 02:15:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 294/870] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:15:52 2023] GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 1090 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:15:54 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:15:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:15:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:15:54 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:15:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] Creating event network-changed:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:15:54 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-10d401fa-3bf8-4d13-a0ef-33168ba2833e req-8dc666c0-6cae-4e0c-be3c-0c2c980a02da service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035939 Apr 12 02:15:54 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 286/871] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:15:54 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:00 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "name": "network-vif-plugged", "status": "completed", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:00 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] Creating event network-vif-plugged:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:16:00 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-9bf1457e-595e-4845-a152-d458b0683c9f req-e4b350b4-fc48-4cab-bfe1-10aa270430bc service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032832 Apr 12 02:16:00 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 292/872] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:00 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "name": "network-vif-plugged", "status": "completed", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] Creating event network-vif-plugged:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:16:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-77938990-eadf-4541-ad7c-280266ab95ae req-a77c2051-9fb2-4f37-9b98-5c34ec085253 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037812 Apr 12 02:16:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 295/873] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:02 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-24116e1d-97b3-4de1-bd7f-1bdd2803c383 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1536 microversion: 2.1 time: 0.289492 Apr 12 02:16:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 287/874] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:17 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1536 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Searching by: {'deleted': False, 'project_id': '71b73a0cde174feeb85558cebb13d137'} {{(pid=72507) get_all /opt/stack/nova/nova/compute/api.py:3019}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72507]: DEBUG nova.compute.multi_cell_list [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=72507) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Apr 12 02:16:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6292e3bb-65f3-455d-9558-157c0e0b9565 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers" status: 200 len: 341 microversion: 2.1 time: 0.170451 Apr 12 02:16:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 293/875] 10.0.0.210 () {62 vars in 1202 bytes} [Wed Apr 12 02:16:17 2023] GET /compute/v2.1/servers => generated 341 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:17 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-80e64d0d-cb0b-4eef-9b4c-5edfa4bc381b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1536 microversion: 2.1 time: 0.161143 Apr 12 02:16:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 296/876] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:17 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1536 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bd53833b-60be-44d0-834d-015f87042307"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-00176c31-f205-4925-a57c-9d4d53ad5f71 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.417502 Apr 12 02:16:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 288/877] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:16:19 2023] POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments => generated 194 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:27 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:27 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-00e76635-b9b9-41e2-a250-14eb0c470587 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1582 microversion: 2.1 time: 0.155463 Apr 12 02:16:27 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 294/878] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:27 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1582 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] Creating event network-changed:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-e78529fa-b8ee-4ae7-bf27-66d1a1687af3 req-74009dd7-1e0b-447b-afe8-fd3ca40ab0c2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104372 Apr 12 02:16:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 297/879] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:29 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ServerBootFromVolumeStableRescueTest-image-968117510"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:30 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-620104799"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:30 user devstack@n-api.service[72507]: INFO nova.network.security_group_api [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Adding security group cac1b3df-4d3e-4ed6-85a1-70e594f87833 to port 2ee24eda-3ac7-48c9-b117-91371281aa33 Apr 12 02:16:30 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5ccec886-3d2f-4e0c-81bc-ad55efc6b442 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb/action" status: 202 len: 52 microversion: 2.87 time: 0.912845 Apr 12 02:16:30 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 289/880] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:16:30 2023] POST /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb/action => generated 52 bytes in 922 msecs (HTTP/1.1 202) 9 headers in 364 bytes (1 switches on core 0) Apr 12 02:16:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-c00a683e-8a5b-4a84-b532-c33b20d8dfad tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action" status: 202 len: 0 microversion: 2.1 time: 0.543022 Apr 12 02:16:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 295/881] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:16:30 2023] POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action => generated 0 bytes in 545 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:16:31 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] Creating event network-changed:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c4a920db-1919-4437-9390-76c967c84be0 req-bf7e20b2-44a2-4eb4-8f19-f3dd0598a882 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037293 Apr 12 02:16:31 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 298/882] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:31 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:31 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6a50ed02-e734-48da-be50-41ede501a0e2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1747 microversion: 2.1 time: 0.198488 Apr 12 02:16:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 290/883] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:31 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1747 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-de8a2a8c-76ce-435b-917d-c2f14ae602bc tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action" status: 202 len: 0 microversion: 2.1 time: 0.161398 Apr 12 02:16:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 296/884] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:16:32 2023] POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:33 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bd96d0f6-ac7a-4cdf-88a9-644a3d164de7 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1749 microversion: 2.1 time: 0.321656 Apr 12 02:16:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 299/885] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:32 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1749 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:16:33 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-unplugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:33 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] Creating event network-vif-unplugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:33 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-217ffa4f-66ac-47ae-8d5d-e68958fad5c1 req-5235125e-a2d0-4b74-8364-6d615527b418 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036852 Apr 12 02:16:33 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 291/886] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:33 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:35 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:35 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:35 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-75fbc7d9-da2f-4740-9c6a-fc0d2b81723f req-86d4f469-66a6-4725-96fa-fef3c63b43e4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031643 Apr 12 02:16:35 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 297/887] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:35 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:36 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d991c5be-4ad9-4386-89df-56fb649b6aca tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 200 len: 1691 microversion: 2.87 time: 0.158227 Apr 12 02:16:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 300/888] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:16:36 2023] GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 1691 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:16:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-5f171822-7dd0-4048-888a-794272b6ecf5 req-3940eea4-7363-40d1-9863-1d405e7263af service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049928 Apr 12 02:16:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 292/889] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:38 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:16:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:16:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3d1c1f13-7894-4de4-b26b-0ddd8975efbb req-d4455868-a594-4a68-87d0-461370143e5d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046005 Apr 12 02:16:40 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 298/890] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:16:40 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:16:42 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:16:42 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8a4e46b5-11bc-4a71-843f-8e6d933b275c tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 200 len: 1442 microversion: 2.1 time: 0.153275 Apr 12 02:16:42 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 301/891] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:16:42 2023] GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 1442 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:23 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-da5f7595-1b0f-4911-9566-b66da82fe268 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1747 microversion: 2.1 time: 0.176089 Apr 12 02:17:23 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 293/892] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:17:23 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1747 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=10 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=25,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerBootFromVolumeStableRescueTest-server-1839044150", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "block_device_mapping_v2": [{"boot_index": "0", "source_type": "blank", "volume_size": 1, "destination_type": "volume"}], "networks": [{"uuid": "68f69bfd-e205-4f13-92e2-346de9c67dd4"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] validate_networks() for [('68f69bfd-e205-4f13-92e2-346de9c67dd4', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for user fe9217f4c1ff428086140b7fd07194eb and project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:17:36 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='blank',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:17:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d47f4bd3-ad21-4f01-95de-09c8d39d73c9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.87 time: 0.538749 Apr 12 02:17:36 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 299/893] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:17:36 2023] POST /compute/v2.1/servers => generated 378 bytes in 540 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:17:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:17:36 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:17:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=22 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-739c367a-eb78-439f-b11a-0c2b9d719b46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1395 microversion: 2.87 time: 0.091929 Apr 12 02:17:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 302/894] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:17:36 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1395 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:39 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] Creating event network-changed:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:17:39 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-aebe4364-ae01-4e25-92da-112f054d276f req-bd20d6da-a3c5-4912-b518-713a60155bf1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033518 Apr 12 02:17:39 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 294/895] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:39 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:42 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6eecf926-3352-4e65-a335-bb4b4ce067f4 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 200 len: 1442 microversion: 2.1 time: 0.174703 Apr 12 02:17:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 300/896] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:17:42 2023] GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 1442 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:17:44 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] Creating event network-changed:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:17:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-302815b2-30c3-40c1-ad75-47b1eb711e66 req-4ce8500d-4328-420e-9abd-f6f1110a8903 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089977 Apr 12 02:17:44 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 303/897] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:44 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1832739782", "imageRef": "", "flavorRef": "42", "key_name": "tempest-TestVolumeBootPattern-1281564430", "security_groups": [{"name": "tempest-secgroup-smoke-150038052"}], "block_device_mapping_v2": [{"uuid": "452f1fa1-24b2-4afe-995f-25e0b08aa4a4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "d395449e-6635-4d1d-afe6-aa4a3c403969"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] validate_networks() for [('d395449e-6635-4d1d-afe6-aa4a3c403969', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Getting quotas for user 69d5d81fffb14b1cab84b6160ceb7dd2 and project bcd685278576490480a97cd2bc5e048b. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:17:47 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='452f1fa1-24b2-4afe-995f-25e0b08aa4a4',volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:17:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-d48beca8-becc-44de-8e01-47c8daea423e tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 403 microversion: 2.1 time: 0.774268 Apr 12 02:17:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 295/898] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:17:47 2023] POST /compute/v2.1/servers => generated 403 bytes in 776 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:17:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-55d42eee-6f93-460b-b1d5-0fbe5a940178 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:17:47 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-55d42eee-6f93-460b-b1d5-0fbe5a940178 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:17:47 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:47 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:48 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-55d42eee-6f93-460b-b1d5-0fbe5a940178 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:17:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-55d42eee-6f93-460b-b1d5-0fbe5a940178 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 200 len: 1090 microversion: 2.1 time: 0.065941 Apr 12 02:17:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 301/899] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:17:47 2023] GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 1090 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:17:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] Creating event network-changed:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:17:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-2940a91e-5d76-4075-9f39-5ed4a3f5050f req-55257c4b-5376-4386-809d-596c2d18ceaf service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036168 Apr 12 02:17:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 304/900] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:50 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:17:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:17:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-9a51e0b8-3f72-4c45-a17a-36e3a2f2ee77 req-2b3a99e4-9f21-4526-b7c2-cf7b3107955a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038622 Apr 12 02:17:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 296/901] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:50 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:17:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:53 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:17:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b6da655c-8ac9-4e4a-9646-bec425e7f87a req-e416e4ce-f9a2-4658-a406-78917263ea33 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034261 Apr 12 02:17:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 302/902] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:17:56 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:56 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:17:56 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-d76bf6e7-a1d1-4193-9664-5175e4a6aa54 req-b6632b01-52ee-475a-b816-54bf08c9770f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041109 Apr 12 02:17:56 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 305/903] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:56 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:17:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:17:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:17:58 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-24d0e338-fae6-4902-b9c0-6811b5cbb7f0 req-577f3fc5-743e-4c38-95ef-e3b86709eab0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099657 Apr 12 02:17:58 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 297/904] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:17:58 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-620104799"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:23 user devstack@n-api.service[72507]: INFO nova.network.security_group_api [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Removing security group cac1b3df-4d3e-4ed6-85a1-70e594f87833 from port 2ee24eda-3ac7-48c9-b117-91371281aa33 Apr 12 02:18:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-380f1909-1126-45f2-ba09-81045211bc49 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action" status: 202 len: 0 microversion: 2.1 time: 0.985628 Apr 12 02:18:24 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 303/905] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:18:23 2023] POST /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/action => generated 0 bytes in 988 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] Creating event network-changed:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:18:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-f1b5d921-05c9-4ed2-ab12-425feb009e96 req-9ad4ecdc-b367-4927-9503-060fab655934 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043880 Apr 12 02:18:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 306/906] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:18:24 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:18:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] Creating event network-changed:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f76d714e-e4f8-4329-b2d6-390b1256b8a6 req-62405f0e-5bac-4b2f-bbed-e15ae5c9528b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.105118 Apr 12 02:18:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 298/907] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:18:25 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:18:25 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lazy-loading 'services' on Instance uuid e6f93d39-5be6-4223-9ac8-362ee29f9e22 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:18:26 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-d2a02da1-2b26-4093-8de6-78abb4438263 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments/bd53833b-60be-44d0-834d-015f87042307" status: 202 len: 0 microversion: 2.1 time: 0.372996 Apr 12 02:18:26 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 304/908] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:18:25 2023] DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments/bd53833b-60be-44d0-834d-015f87042307 => generated 0 bytes in 375 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f0fe7c65-f268-4eaf-9343-1ffed6f5a9cd tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049206 Apr 12 02:18:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 307/909] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:18:26 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:18:27 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:18:27 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0f2d730b-99b1-40ba-b72f-3185be026af7 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1770 microversion: 2.87 time: 0.214771 Apr 12 02:18:27 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 299/910] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:18:27 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1770 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=13,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:18:38 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:18:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0a170ab4-d873-4531-80f4-02d7c45f8d2d tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 200 len: 1443 microversion: 2.1 time: 0.171445 Apr 12 02:18:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 305/911] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:18:38 2023] GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 1443 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b0e02f62-74b4-49eb-b26f-3d5454a98bb9 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046340 Apr 12 02:19:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 308/912] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:19:16 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] HTTP exception thrown: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 is not attached to volume bd53833b-60be-44d0-834d-015f87042307 Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Request method failure captured: Apr 12 02:19:16 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments/bd53833b-60be-44d0-834d-015f87042307 HTTP/1.1 Apr 12 02:19:16 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:19:16 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:19:16 user devstack@n-api.service[72505]: Connection: close Apr 12 02:19:16 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:19:16 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:19:16 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:19:16 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:19:16 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhP4TNLkskTT1COUrpahKnBl3GpBpPJj50ImbBXo5pNdCc94W-ZWRXHib94Bpa-qBLK6M1JDI2HegA1FZqtwennGq0DrpY5oyu8umFDx5MAx9lhOdL1vcIu_qF9-XQk9pDSjPkgt6ZlEDJtBy-NrPIVXOnKtaspJBprymUWyhNiyWRU Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Project-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Tenant: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Tenant-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-User: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:19:16 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:19:16 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:19:16 user devstack@n-api.service[72505]: X-User-Id: 88570fbc28654337a032da5fa1f58814 Apr 12 02:19:16 user devstack@n-api.service[72505]: X-User-Name: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:19:16 user devstack@n-api.service[72505]: method: > Apr 12 02:19:16 user devstack@n-api.service[72505]: exception: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 is not attached to volume bd53833b-60be-44d0-834d-015f87042307 Apr 12 02:19:16 user devstack@n-api.service[72505]: action_args: {'server_id': 'e6f93d39-5be6-4223-9ac8-362ee29f9e22', 'id': 'bd53833b-60be-44d0-834d-015f87042307'} Apr 12 02:19:16 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id bd53833b-60be-44d0-834d-015f87042307. Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 is not attached to volume bd53833b-60be-44d0-834d-015f87042307 Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 is not attached to volume bd53833b-60be-44d0-834d-015f87042307 Apr 12 02:19:16 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:19:16 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Returning 404 to user: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 is not attached to volume bd53833b-60be-44d0-834d-015f87042307 {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:19:16 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-c85f42f9-9e62-436e-95cc-2132629b6f09 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments/bd53833b-60be-44d0-834d-015f87042307" status: 404 len: 154 microversion: 2.1 time: 0.123575 Apr 12 02:19:16 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 300/913] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:19:16 2023] DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22/os-volume_attachments/bd53833b-60be-44d0-834d-015f87042307 => generated 154 bytes in 125 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:18 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:19:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-de49922b-96e2-49d8-ae12-cf1d8b429465 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 204 len: 0 microversion: 2.1 time: 0.112548 Apr 12 02:19:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 306/914] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:19:18 2023] DELETE /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:18 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b59b35e9-2b37-42c2-9bbb-3cfbdd25a2c2 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 200 len: 1542 microversion: 2.1 time: 0.183314 Apr 12 02:19:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 309/915] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:19:18 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 1542 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:19:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-unplugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] Creating event network-vif-unplugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:19:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-9f021dbf-f22e-43c8-ac6c-12fefbc6c9b1 req-85cebfb0-4862-47c4-9465-6047c3b4d1a2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035269 Apr 12 02:19:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 301/916] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:18 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:20 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-deleted", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:20 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] Creating event network-vif-deleted:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:19:20 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-384cce81-6206-496e-b8cc-b2d6bc23ef7e req-98fd91c1-381c-47d0-8ea6-d839e8d4d76c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030376 Apr 12 02:19:20 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 307/917] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:20 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6f93d39-5be6-4223-9ac8-362ee29f9e22", "name": "network-vif-plugged", "status": "completed", "tag": "2ee24eda-3ac7-48c9-b117-91371281aa33"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] Creating event network-vif-plugged:2ee24eda-3ac7-48c9-b117-91371281aa33 for instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 on user Apr 12 02:19:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-721140f5-3759-41f5-b64e-3002bbf7229c req-f993662a-72fd-4566-8ca8-8f36de58856a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.028976 Apr 12 02:19:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 310/918] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:21 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"rescue": {"rescue_image_ref": "fea148e9-4d9a-4c1c-9836-28e2777cc93e"}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:27 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lazy-loading 'flavor' on Instance uuid 811c4fb7-d984-461e-9190-4d0e9ec345c2 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:19:28 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a0c9158f-8f98-405b-932a-1fe3f45f2d15 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2/action" status: 200 len: 29 microversion: 2.87 time: 0.547698 Apr 12 02:19:28 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 302/919] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:19:27 2023] POST /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2/action => generated 29 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:28 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-ce66dff1-f990-482e-96a7-7899b5839be9 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1776 microversion: 2.87 time: 0.239618 Apr 12 02:19:28 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 308/920] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:19:28 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1776 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:19:28 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-unplugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] Creating event network-vif-unplugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:19:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-3822dd1a-89fb-45b2-b6f0-cea46930f34d req-de252e75-ef5c-4ffb-a674-483a849beae3 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031995 Apr 12 02:19:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 311/921] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:28 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:19:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-dc122366-2de2-42a9-be1e-cc95b311ba47 req-e301e6c8-09e8-4adb-bde0-c17b1c18b1d6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038823 Apr 12 02:19:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 303/922] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:31 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:33 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:33 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:19:33 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-53e89305-7f79-4936-8548-5ff128106b4e req-e1b41e61-bf3e-4db1-a062-1d18c91a9561 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030071 Apr 12 02:19:33 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 309/923] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:33 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:35 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:19:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-515d86c6-13a6-4a82-9943-f7d4ff4e04cc req-b190f95a-4576-48ee-ab76-4f8048b476ad service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030093 Apr 12 02:19:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 312/924] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:35 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:38 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-18762d52-24f7-4135-a359-f8a78ab49b84 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 200 len: 1443 microversion: 2.1 time: 0.211563 Apr 12 02:19:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 304/925] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:19:38 2023] GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 1443 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] Creating event network-changed:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:40 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3f0c123d-724c-4dcc-81d2-936a922a2f53 req-7473c943-37dc-4cbb-b97c-05ce8a215e24 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031712 Apr 12 02:19:40 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 310/926] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:40 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:40 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:19:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] Creating event network-changed:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a68ed19d-200d-4ac5-a44c-1a92970b097d req-12c1832e-7d3e-44e8-bf54-47ee979b0d38 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062146 Apr 12 02:19:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 305/927] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:40 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-657dfa3b-2a04-45cd-a33e-3dceece32e42 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 204 len: 0 microversion: 2.1 time: 0.168034 Apr 12 02:19:40 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 313/928] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:19:40 2023] DELETE /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:40 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:41 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:19:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-408be3dc-fecd-4ffe-921f-7a643c8a5ec7 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 200 len: 1568 microversion: 2.1 time: 0.729805 Apr 12 02:19:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 311/929] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:19:40 2023] GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 1568 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:19:42 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-unplugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}, {"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}, {"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}, {"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:42 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Creating event network-vif-unplugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:42 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a98afb7b-bcc6-44ab-b84d-040e2fe573f4 req-a74ab2b8-431a-42dd-98bb-5f89b9ab1577 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 698 microversion: 2.1 time: 0.033414 Apr 12 02:19:42 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 306/930] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:42 2023] POST /compute/v2.1/os-server-external-events => generated 698 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:19:44 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-unplugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}, {"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-plugged", "status": "completed", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}, {"server_uuid": "7280568b-5a65-43ab-b645-605ea5652324", "name": "network-vif-deleted", "tag": "26028cc6-2fb8-44a6-8ecb-26e121018cc1"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:19:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:19:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:19:44 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:19:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Creating event network-vif-unplugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Creating event network-vif-plugged:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] Creating event network-vif-deleted:26028cc6-2fb8-44a6-8ecb-26e121018cc1 for instance 7280568b-5a65-43ab-b645-605ea5652324 on user Apr 12 02:19:44 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-fcac2dc9-fa0e-4063-98c3-349b102584e5 req-6feab767-2fa6-4717-849a-25a2acd62980 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 527 microversion: 2.1 time: 0.030167 Apr 12 02:19:44 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 314/931] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:19:44 2023] POST /compute/v2.1/os-server-external-events => generated 527 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=33,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: e6f93d39-5be6-4223-9ac8-362ee29f9e22] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] HTTP exception thrown: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Request method failure captured: Apr 12 02:20:09 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 HTTP/1.1 Apr 12 02:20:09 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:20:09 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:20:09 user devstack@n-api.service[72507]: Connection: close Apr 12 02:20:09 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:20:09 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:20:09 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:20:09 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:20:09 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhP4TNLkskTT1COUrpahKnBl3GpBpPJj50ImbBXo5pNdCc94W-ZWRXHib94Bpa-qBLK6M1JDI2HegA1FZqtwennGq0DrpY5oyu8umFDx5MAx9lhOdL1vcIu_qF9-XQk9pDSjPkgt6ZlEDJtBy-NrPIVXOnKtaspJBprymUWyhNiyWRU Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Project-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Tenant: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Tenant-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-User: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:20:09 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:20:09 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:20:09 user devstack@n-api.service[72507]: X-User-Id: 88570fbc28654337a032da5fa1f58814 Apr 12 02:20:09 user devstack@n-api.service[72507]: X-User-Name: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:20:09 user devstack@n-api.service[72507]: method: > Apr 12 02:20:09 user devstack@n-api.service[72507]: exception: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. Apr 12 02:20:09 user devstack@n-api.service[72507]: action_args: {'id': 'e6f93d39-5be6-4223-9ac8-362ee29f9e22'} Apr 12 02:20:09 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. Apr 12 02:20:09 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Returning 404 to user: Instance e6f93d39-5be6-4223-9ac8-362ee29f9e22 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:20:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-151be722-1e52-4696-94b6-672e85c0f84b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22" status: 404 len: 111 microversion: 2.1 time: 0.197380 Apr 12 02:20:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 312/932] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:20:09 2023] GET /compute/v2.1/servers/e6f93d39-5be6-4223-9ac8-362ee29f9e22 => generated 111 bytes in 199 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Apr 12 02:20:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:09 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=37,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ea984d5f-05f2-4018-b05a-456ce50d5604 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '.version_select at 0x7fb2033f5b40>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:09 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ea984d5f-05f2-4018-b05a-456ce50d5604 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1687627615" status: 202 len: 0 microversion: 2.1 time: 0.036406 Apr 12 02:20:09 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 307/933] 10.0.0.210 () {62 vars in 1349 bytes} [Wed Apr 12 02:20:09 2023] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1687627615 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:20:11 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:11 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=35,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:11 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-36ccf698-e026-4aa7-9241-ae6dad91d8ca tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: .version_select at 0x7fb203199120>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-930950318", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGRiQvqoiry2HzQuAwe/Vgtc//3bwoA4G8ab1sp4euF4u2N4/2/yd2ZM5F60Io7Szs5Bjs3LQMxAsDKjmtd2F35MDF3acLGrWxj7gQuIphjSqqZuF6DF4p1A0yUIs40Gkg=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:11 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-36ccf698-e026-4aa7-9241-ae6dad91d8ca tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for project 71b73a0cde174feeb85558cebb13d137. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:20:11 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-36ccf698-e026-4aa7-9241-ae6dad91d8ca tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for user 88570fbc28654337a032da5fa1f58814 and project 71b73a0cde174feeb85558cebb13d137. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:20:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-36ccf698-e026-4aa7-9241-ae6dad91d8ca tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.058355 Apr 12 02:20:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 315/934] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:20:11 2023] POST /compute/v2.1/os-keypairs => generated 402 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-47460833", "imageRef": "b6ae0cea-7c74-495d-ae4c-3c01f51c692e", "flavorRef": "42", "key_name": "tempest-TestMinimumBasicScenario-930950318", "networks": [{"uuid": "d62e0f89-6046-4a5f-9fda-0c742420c904"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] validate_networks() for [('d62e0f89-6046-4a5f-9fda-0c742420c904', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:20:11 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for project 71b73a0cde174feeb85558cebb13d137. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Getting quotas for user 88570fbc28654337a032da5fa1f58814 and project 71b73a0cde174feeb85558cebb13d137. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:20:12 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b6ae0cea-7c74-495d-ae4c-3c01f51c692e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:20:12 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6d87491b-1d57-4a74-be39-7a5bfefc26aa tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.1 time: 0.471778 Apr 12 02:20:12 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 313/935] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:20:11 2023] POST /compute/v2.1/servers => generated 378 bytes in 473 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:20:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1c15e99b-2778-4810-9ddb-a8a998bf9523 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:12 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1c15e99b-2778-4810-9ddb-a8a998bf9523 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:12 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-1c15e99b-2778-4810-9ddb-a8a998bf9523 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:20:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1c15e99b-2778-4810-9ddb-a8a998bf9523 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1250 microversion: 2.1 time: 0.116852 Apr 12 02:20:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 308/936] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:20:12 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1250 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:20:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] Creating event network-changed:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:20:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8aad7f5c-017a-4282-a273-c5602c3890b3 req-0b18b39a-7c77-4c78-b5a5-397c85c7aa99 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037304 Apr 12 02:20:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 316/937] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:20:14 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:20:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b0f3a69f-e70f-4b3e-b7cc-0dab258ef9a4 req-66a4d938-dc15-472c-a861-948cb75c6e37 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031838 Apr 12 02:20:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 314/938] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:20:16 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:20:18 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:20:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-32800895-940c-4f55-93eb-7617cc353eaf tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1756 microversion: 2.87 time: 0.208643 Apr 12 02:20:18 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 309/939] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:20:18 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1756 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:20:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:20:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ff148665-aa8d-4ad3-a0cd-2cd26ddbbc2d req-54e60d71-5986-4e30-98fa-2978746131c2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032579 Apr 12 02:20:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 317/940] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:20:18 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=19,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 7280568b-5a65-43ab-b645-605ea5652324] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:20:31 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 HTTP/1.1 Apr 12 02:20:31 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:20:31 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:20:31 user devstack@n-api.service[72507]: Connection: close Apr 12 02:20:31 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:20:31 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:20:31 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:20:31 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:20:31 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:20:31 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:20:31 user devstack@n-api.service[72507]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:20:31 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:20:31 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:20:31 user devstack@n-api.service[72507]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:20:31 user devstack@n-api.service[72507]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:20:31 user devstack@n-api.service[72507]: method: > Apr 12 02:20:31 user devstack@n-api.service[72507]: exception: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. Apr 12 02:20:31 user devstack@n-api.service[72507]: action_args: {'id': '7280568b-5a65-43ab-b645-605ea5652324'} Apr 12 02:20:31 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. Apr 12 02:20:31 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:20:31 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 7280568b-5a65-43ab-b645-605ea5652324 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:20:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4bf4aa03-ed84-42e1-b498-9412216619bf tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324" status: 404 len: 111 microversion: 2.1 time: 0.040190 Apr 12 02:20:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 315/941] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:20:31 2023] GET /compute/v2.1/servers/7280568b-5a65-43ab-b645-605ea5652324 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=11,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=15,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] Creating event network-changed:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:20:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-b5b0a51b-a9d8-4a60-b7f6-ea0091f316ac req-b87dbd3e-fcd3-4f6e-a9e0-6bb6b745a029 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039779 Apr 12 02:20:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 318/942] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:20:34 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:20:34 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:20:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4fbe1f0e-3350-43d2-8fa3-954a5b105c47 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 204 len: 0 microversion: 2.1 time: 0.139324 Apr 12 02:20:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 310/943] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:20:34 2023] DELETE /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:34 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69d5d81fffb14b1cab84b6160ceb7dd2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcd685278576490480a97cd2bc5e048b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:20:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a35852d1-c39b-45bd-984b-048aa3ec021b tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 200 len: 1566 microversion: 2.1 time: 0.324187 Apr 12 02:20:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 316/944] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:20:34 2023] GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 1566 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:20:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "name": "network-vif-unplugged", "status": "completed", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}, {"server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "name": "network-vif-plugged", "status": "completed", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}, {"server_uuid": "8553081b-42a3-45c9-a6f6-248315547366", "name": "network-vif-deleted", "tag": "f3bfac63-9965-4cc9-9c67-91d735cdcc43"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:20:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:20:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:20:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:20:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Creating event network-vif-unplugged:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:20:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Creating event network-vif-plugged:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:20:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] Creating event network-vif-deleted:f3bfac63-9965-4cc9-9c67-91d735cdcc43 for instance 8553081b-42a3-45c9-a6f6-248315547366 on user Apr 12 02:20:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-31c01f9b-57ee-4662-be65-47b75758477d req-24986bdb-65c2-4ef8-9d75-35b78f526d0d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 527 microversion: 2.1 time: 0.032091 Apr 12 02:20:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 319/945] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:20:36 2023] POST /compute/v2.1/os-server-external-events => generated 527 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=15,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=7,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:02 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:21:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-de8d91b2-7d74-42ba-83fb-204039fcb018 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1533 microversion: 2.1 time: 0.190012 Apr 12 02:21:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 311/946] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:21:02 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1533 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:18 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3c2cefa8-cc3e-47a7-a9f9-c1d324cf14d3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2/action" status: 202 len: 0 microversion: 2.87 time: 0.094722 Apr 12 02:21:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 317/947] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:21:18 2023] POST /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2/action => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:18 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:19 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:21:19 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:21:19 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:21:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5f645626-f35b-426e-8c6b-6c418790845f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1764 microversion: 2.87 time: 0.603719 Apr 12 02:21:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 320/948] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:21:18 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1764 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-unplugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] Creating event network-vif-unplugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:21:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8cf2cf74-db28-461d-a766-8f46d0edf277 req-96a2d554-5f70-4fd7-a998-b272978552ce service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035774 Apr 12 02:21:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 312/949] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:21:19 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:21:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}, {"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:21:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:21:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:21:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-1625a710-9bc8-45b0-b1b5-01fd635873cd req-c9b754d3-676e-45f8-aabe-b9cdbeecc9e2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.034855 Apr 12 02:21:22 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 318/950] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:21:21 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:21:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:21:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:21:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-d8146e8c-8bf9-434f-a976-6485fc6ff458 req-4d29478d-73b1-427d-bab8-e975b304d54c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050066 Apr 12 02:21:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 321/951] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:21:24 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: 8553081b-42a3-45c9-a6f6-248315547366] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:24 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:21:25 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 HTTP/1.1 Apr 12 02:21:25 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:21:25 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:21:25 user devstack@n-api.service[72505]: Connection: close Apr 12 02:21:25 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:21:25 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:21:25 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:21:25 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:21:25 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72505]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72505]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:21:25 user devstack@n-api.service[72505]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72505]: method: > Apr 12 02:21:25 user devstack@n-api.service[72505]: exception: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72505]: action_args: {'id': '8553081b-42a3-45c9-a6f6-248315547366'} Apr 12 02:21:25 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance 8553081b-42a3-45c9-a6f6-248315547366 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:21:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-99bde6a1-a0b7-452a-b537-19b30b803ab3 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366" status: 404 len: 111 microversion: 2.1 time: 0.073225 Apr 12 02:21:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 313/952] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:21:24 2023] GET /compute/v2.1/servers/8553081b-42a3-45c9-a6f6-248315547366 => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:21:25 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 HTTP/1.1 Apr 12 02:21:25 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:21:25 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:21:25 user devstack@n-api.service[72507]: Connection: close Apr 12 02:21:25 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:21:25 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:21:25 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:21:25 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:21:25 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72507]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72507]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:21:25 user devstack@n-api.service[72507]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72507]: method: > Apr 12 02:21:25 user devstack@n-api.service[72507]: exception: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72507]: action_args: {'id': 'fc88ac73-1963-4055-bd73-7a51c0850528'} Apr 12 02:21:25 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:21:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a5356e46-3de6-4f4f-86aa-5e44ceb550a0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 404 len: 111 microversion: 2.1 time: 0.032381 Apr 12 02:21:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 319/953] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:21:25 2023] DELETE /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] [instance: fc88ac73-1963-4055-bd73-7a51c0850528] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:21:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] HTTP exception thrown: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Request method failure captured: Apr 12 02:21:25 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 HTTP/1.1 Apr 12 02:21:25 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:21:25 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:21:25 user devstack@n-api.service[72506]: Connection: close Apr 12 02:21:25 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:21:25 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:21:25 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:21:25 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:21:25 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhGzV-nBeYGuFyP7FBcm1a0pM4Mq7tUb41V5_B30rCazg5kdqe55rg1uRFgORiwEWuwmzxTS0XVlYqYx1YjTXApHzj5khUcGw9dW1zBcUIkdFPQwiOQmfppaTqgtjatlq72WQQAcUoyIgnfyJ-outzyPzekRzpdUil7lAmDdHLdFPPM Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Project-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/bcd685278576490480a97cd2bc5e048b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Tenant: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Tenant-Id: bcd685278576490480a97cd2bc5e048b Apr 12 02:21:25 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestVolumeBootPattern-2043953815 Apr 12 02:21:25 user devstack@n-api.service[72506]: X-User: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:21:25 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:21:25 user devstack@n-api.service[72506]: X-User-Id: 69d5d81fffb14b1cab84b6160ceb7dd2 Apr 12 02:21:25 user devstack@n-api.service[72506]: X-User-Name: tempest-TestVolumeBootPattern-2043953815-project-member Apr 12 02:21:25 user devstack@n-api.service[72506]: method: > Apr 12 02:21:25 user devstack@n-api.service[72506]: exception: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72506]: action_args: {'id': 'fc88ac73-1963-4055-bd73-7a51c0850528'} Apr 12 02:21:25 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. Apr 12 02:21:25 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:21:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Returning 404 to user: Instance fc88ac73-1963-4055-bd73-7a51c0850528 could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:21:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-eb0733c4-60b4-4393-b492-84a664e53047 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528" status: 404 len: 111 microversion: 2.1 time: 0.032008 Apr 12 02:21:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 322/954] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:21:25 2023] GET /compute/v2.1/servers/fc88ac73-1963-4055-bd73-7a51c0850528 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:21:26 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7586c27c-03f1-4761-8fcd-e084afd2cfb0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] Calling method '.version_select at 0x7fb2030912d0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:21:26 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7586c27c-03f1-4761-8fcd-e084afd2cfb0 tempest-TestVolumeBootPattern-2043953815 tempest-TestVolumeBootPattern-2043953815-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1281564430" status: 202 len: 0 microversion: 2.1 time: 0.023931 Apr 12 02:21:26 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 314/955] 10.0.0.210 () {62 vars in 1340 bytes} [Wed Apr 12 02:21:26 2023] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1281564430 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=14,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:02 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:02 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-78e4fbee-0196-49af-85a3-31eb7eb2f35e tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1533 microversion: 2.1 time: 0.165953 Apr 12 02:22:02 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 320/956] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:22:02 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1533 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Searching by: {'deleted': False, 'project_id': '71b73a0cde174feeb85558cebb13d137'} {{(pid=72506) get_all /opt/stack/nova/nova/compute/api.py:3019}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=10 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:02 user devstack@n-api.service[72506]: DEBUG nova.compute.multi_cell_list [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=72506) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Apr 12 02:22:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-59b7c458-faac-4b75-b732-4548627a7ec4 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers" status: 200 len: 339 microversion: 2.1 time: 0.170101 Apr 12 02:22:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 323/957] 10.0.0.210 () {62 vars in 1202 bytes} [Wed Apr 12 02:22:02 2023] GET /compute/v2.1/servers => generated 339 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=7,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:03 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-2454dbc8-b0e9-4ee1-840a-2682d39c6b76 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1533 microversion: 2.1 time: 0.180560 Apr 12 02:22:03 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 315/958] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:22:02 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1533 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6f29ab89-68a0-4841-9a41-173da8724057"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:04 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:04 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bf4a7626-f491-41fd-b89a-d0b94f373368 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.312262 Apr 12 02:22:04 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 321/959] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:22:04 2023] POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments => generated 194 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:09 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cb66c1fa-d2c6-4f71-b317-3a6640e65c4f tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1770 microversion: 2.87 time: 0.215864 Apr 12 02:22:09 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 324/960] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:22:09 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1770 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:12 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-26918ee0-b149-488b-b699-78efc0e9bf7a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1579 microversion: 2.1 time: 0.169614 Apr 12 02:22:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 316/961] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:22:12 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1579 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:15 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] Creating event network-changed:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:15 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7180d396-1570-42c2-bf27-e8d057289878 req-b00b8429-5a48-4366-b641-4bab9af804f4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050846 Apr 12 02:22:15 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 322/962] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:22:15 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1989521032"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:16 user devstack@n-api.service[72506]: INFO nova.network.security_group_api [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Adding security group ebc27437-1909-4e10-85a1-c66054cd3b87 to port 00eb0800-cff6-4e7f-aa82-0103d8cc5153 Apr 12 02:22:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-ab9f7971-d868-498b-81d2-e651b07d2d66 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action" status: 202 len: 0 microversion: 2.1 time: 0.983030 Apr 12 02:22:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 325/963] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:22:16 2023] POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action => generated 0 bytes in 984 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] Creating event network-changed:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-0790371b-2ac5-4748-b4bb-0f4f82d19d03 req-bb98a1df-8a19-4b47-a66a-3118cbeb37ec service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080107 Apr 12 02:22:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 323/964] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:22:17 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-86c7a91e-c292-44df-bca6-135c8a5ef352 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1745 microversion: 2.1 time: 0.234332 Apr 12 02:22:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 317/965] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:22:17 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1745 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-9892ab58-98d1-415d-b9ce-66da4aaff14f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action" status: 202 len: 0 microversion: 2.1 time: 0.098859 Apr 12 02:22:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 326/966] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:22:17 2023] POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action => generated 0 bytes in 100 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:18 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:22:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f85843f5-991f-4556-b4d2-5cfc85b0113b tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1737 microversion: 2.1 time: 0.217821 Apr 12 02:22:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 324/967] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:22:17 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1737 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:22:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-unplugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:21 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] Creating event network-vif-unplugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f5ed7446-409f-421f-aebf-d0574ef852f5 req-d2068c33-70a6-4dba-8334-3bfbf2a9bd69 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.033791 Apr 12 02:22:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 318/968] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:22:21 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}, {"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-937873de-1402-4eae-bbfc-4e5d01b957ae req-aff18b18-de16-44d8-8e37-259f8abf793a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.037938 Apr 12 02:22:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 327/969] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:22:23 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:22:25 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:22:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:22:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:22:25 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:22:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:22:25 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-0b3ac299-5f05-4329-8fc8-b636b70a3024 req-8d5ff41f-281f-4331-a678-4ff8b4804dec service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030150 Apr 12 02:22:25 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 325/970] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:22:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:23:08 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=20 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:08 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:23:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b7ec98f3-d869-4679-980e-e79db81642b6 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1745 microversion: 2.1 time: 0.170072 Apr 12 02:23:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 319/971] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:23:08 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1745 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:23:09 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:09 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerBootFromVolumeStableRescueTest-server-1841847417", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "networks": [{"uuid": "68f69bfd-e205-4f13-92e2-346de9c67dd4"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] validate_networks() for [('68f69bfd-e205-4f13-92e2-346de9c67dd4', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for user fe9217f4c1ff428086140b7fd07194eb and project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:23:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:23:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b6113477-9dd5-4db2-bc92-9586dadde0eb tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.87 time: 0.607275 Apr 12 02:23:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 328/972] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:23:09 2023] POST /compute/v2.1/servers => generated 378 bytes in 610 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:23:10 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:23:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-0e9cec42-8c76-4c14-a9b8-0a272b91deb3 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 200 len: 1395 microversion: 2.87 time: 0.096191 Apr 12 02:23:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 326/973] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:23:10 2023] GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 1395 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:23:12 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:23:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:12 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] Creating event network-changed:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:23:12 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-1c108a54-fdcc-47e5-b740-0252875d170f req-ef14be87-64a9-41f5-bb11-3cc6e0d95d26 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032019 Apr 12 02:23:12 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 320/974] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:23:12 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:23:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "name": "network-vif-plugged", "status": "completed", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:23:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:15 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] Creating event network-vif-plugged:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:23:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-64188b9e-3976-40bc-ad98-25c90976648a req-72202fe4-3459-43ac-93a8-2b252a407bce service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.203777 Apr 12 02:23:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 329/975] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:23:14 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:23:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "name": "network-vif-plugged", "status": "completed", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:23:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:23:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:23:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:23:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] Creating event network-vif-plugged:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:23:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-aa57a443-180e-4bc2-ac86-9a99d76490b5 req-42f03dee-1cfd-4ac7-9e62-128b3fb3c338 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029235 Apr 12 02:23:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 327/976] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:23:17 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:00 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 36248. Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:24:00 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:24:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-66ca3184-7ac4-40f1-8881-2a26e52f4216 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 200 len: 1692 microversion: 2.87 time: 0.173260 Apr 12 02:24:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 321/977] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:24:00 2023] GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 1692 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1989521032"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=15,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:08 user devstack@n-api.service[72506]: INFO nova.network.security_group_api [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Removing security group ebc27437-1909-4e10-85a1-c66054cd3b87 from port 00eb0800-cff6-4e7f-aa82-0103d8cc5153 Apr 12 02:24:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c4fc2a71-6651-480c-82b8-5758985df32d tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action" status: 202 len: 0 microversion: 2.1 time: 0.666921 Apr 12 02:24:09 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 330/978] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:24:08 2023] POST /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/action => generated 0 bytes in 669 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:09 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] Creating event network-changed:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:24:09 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-3643366b-06ac-4fab-ba60-fc786105fc8c req-c24487b8-d9bb-4aec-b436-e69b8ce72587 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034919 Apr 12 02:24:09 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 328/979] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:24:09 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:24:09 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:24:09 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:24:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:09 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:24:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] Creating event network-changed:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:24:10 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-51a41141-20aa-4e13-b34d-f14f315bcea6 req-643583bb-1116-49a5-b52c-aa3f967954f9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097394 Apr 12 02:24:10 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 331/980] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:24:09 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:24:10 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lazy-loading 'services' on Instance uuid 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:24:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b70ffd83-c7d5-4f41-8b70-9c58396d11ab tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057" status: 202 len: 0 microversion: 2.1 time: 0.272049 Apr 12 02:24:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 322/981] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:24:09 2023] DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057 => generated 0 bytes in 279 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:24:10 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:24:10 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-da1dc3d1-f343-4522-bbd3-c7d9fe1a3a60 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046263 Apr 12 02:24:10 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 329/982] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:24:10 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:00 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-fe60fe85-b33b-4983-bec8-21b4d3ab997a tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048201 Apr 12 02:25:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 332/983] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:25:00 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:25:00 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:25:00 user devstack@n-api.service[72505]: Wed Apr 12 02:25:00 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057 (ip 10.0.0.210) !!! Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] HTTP exception thrown: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f is not attached to volume 6f29ab89-68a0-4841-9a41-173da8724057 Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Request method failure captured: Apr 12 02:25:00 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057 HTTP/1.1 Apr 12 02:25:00 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:25:00 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:25:00 user devstack@n-api.service[72505]: Connection: close Apr 12 02:25:00 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:25:00 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:25:00 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:25:00 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:25:00 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhP4TNLkskTT1COUrpahKnBl3GpBpPJj50ImbBXo5pNdCc94W-ZWRXHib94Bpa-qBLK6M1JDI2HegA1FZqtwennGq0DrpY5oyu8umFDx5MAx9lhOdL1vcIu_qF9-XQk9pDSjPkgt6ZlEDJtBy-NrPIVXOnKtaspJBprymUWyhNiyWRU Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Project-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Tenant: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Tenant-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-User: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:25:00 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:25:00 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:25:00 user devstack@n-api.service[72505]: X-User-Id: 88570fbc28654337a032da5fa1f58814 Apr 12 02:25:00 user devstack@n-api.service[72505]: X-User-Name: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:25:00 user devstack@n-api.service[72505]: method: > Apr 12 02:25:00 user devstack@n-api.service[72505]: exception: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f is not attached to volume 6f29ab89-68a0-4841-9a41-173da8724057 Apr 12 02:25:00 user devstack@n-api.service[72505]: action_args: {'server_id': '5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f', 'id': '6f29ab89-68a0-4841-9a41-173da8724057'} Apr 12 02:25:00 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 6f29ab89-68a0-4841-9a41-173da8724057. Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f is not attached to volume 6f29ab89-68a0-4841-9a41-173da8724057 Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f is not attached to volume 6f29ab89-68a0-4841-9a41-173da8724057 Apr 12 02:25:00 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:25:00 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Returning 404 to user: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f is not attached to volume 6f29ab89-68a0-4841-9a41-173da8724057 {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:25:00 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8d7d15a2-9a73-4bec-bfa3-e17a9f0fed16 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057" status: 404 len: 154 microversion: 2.1 time: 0.134911 Apr 12 02:25:00 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 323/984] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:25:00 2023] DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f/os-volume_attachments/6f29ab89-68a0-4841-9a41-173da8724057 => generated 154 bytes in 137 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ServerBootFromVolumeStableRescueTest-image-1300320607"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:01 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:01 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-053fd2ff-fc83-49b2-904f-2fb19732cec1 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8/action" status: 202 len: 52 microversion: 2.87 time: 0.806769 Apr 12 02:25:01 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 330/985] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:25:01 2023] POST /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8/action => generated 52 bytes in 808 msecs (HTTP/1.1 202) 9 headers in 364 bytes (1 switches on core 0) Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:02 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:25:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5ae9755f-7269-4478-9c8f-212b2b59d199 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 204 len: 0 microversion: 2.1 time: 0.132105 Apr 12 02:25:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 333/986] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:25:02 2023] DELETE /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:03 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88570fbc28654337a032da5fa1f58814', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '71b73a0cde174feeb85558cebb13d137', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:25:03 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-87ba9a1c-a5a9-4b82-b96e-396505d98410 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 200 len: 1539 microversion: 2.1 time: 0.204559 Apr 12 02:25:03 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 324/987] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:25:02 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 1539 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:25:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-unplugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:25:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] Creating event network-vif-unplugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:25:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b06966d4-484b-45fa-b1b1-2fd1e2ac323a req-22f44b71-d4a8-40f6-8e8f-26a808e06255 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047580 Apr 12 02:25:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 331/988] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:25:03 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:25:04 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-deleted", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:25:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:04 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] Creating event network-vif-deleted:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:25:04 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-b5ba1d53-2b62-40db-9ce0-d2d1198ef708 req-8f928727-0861-456c-97d6-a4ee7e0913d1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032638 Apr 12 02:25:04 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 334/989] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:25:04 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:25:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f", "name": "network-vif-plugged", "status": "completed", "tag": "00eb0800-cff6-4e7f-aa82-0103d8cc5153"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:25:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:05 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] Creating event network-vif-plugged:00eb0800-cff6-4e7f-aa82-0103d8cc5153 for instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f on user Apr 12 02:25:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-cbc31298-c6d7-4d2f-a717-48313e7931b5 req-bae54445-833b-4bb8-9c1a-effd5bdb3aa8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035266 Apr 12 02:25:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 325/990] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:25:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:25:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:25:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-4ff59bd4-1ea8-4ead-abc6-6168aa78edd0 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 200 len: 1692 microversion: 2.87 time: 0.180228 Apr 12 02:25:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 332/991] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:25:07 2023] GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 1692 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=11,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=23,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:53 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:25:53 user devstack@n-api.service[72506]: Wed Apr 12 02:25:53 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f (ip 10.0.0.210) !!! Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] [instance: 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:25:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] HTTP exception thrown: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Request method failure captured: Apr 12 02:25:53 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f HTTP/1.1 Apr 12 02:25:53 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:25:53 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:25:53 user devstack@n-api.service[72506]: Connection: close Apr 12 02:25:53 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:25:53 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:25:53 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:25:53 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:25:53 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhP4TNLkskTT1COUrpahKnBl3GpBpPJj50ImbBXo5pNdCc94W-ZWRXHib94Bpa-qBLK6M1JDI2HegA1FZqtwennGq0DrpY5oyu8umFDx5MAx9lhOdL1vcIu_qF9-XQk9pDSjPkgt6ZlEDJtBy-NrPIVXOnKtaspJBprymUWyhNiyWRU Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Project-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Project-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/71b73a0cde174feeb85558cebb13d137"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Tenant: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Tenant-Id: 71b73a0cde174feeb85558cebb13d137 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-TestMinimumBasicScenario-934200257 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-User: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:25:53 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:25:53 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:25:53 user devstack@n-api.service[72506]: X-User-Id: 88570fbc28654337a032da5fa1f58814 Apr 12 02:25:53 user devstack@n-api.service[72506]: X-User-Name: tempest-TestMinimumBasicScenario-934200257-project-member Apr 12 02:25:53 user devstack@n-api.service[72506]: method: > Apr 12 02:25:53 user devstack@n-api.service[72506]: exception: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. Apr 12 02:25:53 user devstack@n-api.service[72506]: action_args: {'id': '5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f'} Apr 12 02:25:53 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. Apr 12 02:25:53 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:25:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Returning 404 to user: Instance 5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:25:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-237547e1-76c1-41c8-b30e-5106061be668 tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f" status: 404 len: 111 microversion: 2.1 time: 0.198769 Apr 12 02:25:53 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 335/992] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:25:53 2023] GET /compute/v2.1/servers/5c91bc86-0510-4114-a2ba-9e3f6b3a5e6f => generated 111 bytes in 201 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:25:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=14 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:25:53 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:25:53 user devstack@n-api.service[72505]: Wed Apr 12 02:25:53 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-930950318 (ip 10.0.0.210) !!! Apr 12 02:25:53 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-07382459-718e-40f6-a0f8-135c0b395e0f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] Calling method '.version_select at 0x7fb2033f55a0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:25:53 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-07382459-718e-40f6-a0f8-135c0b395e0f tempest-TestMinimumBasicScenario-934200257 tempest-TestMinimumBasicScenario-934200257-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-930950318" status: 202 len: 0 microversion: 2.1 time: 0.045301 Apr 12 02:25:53 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 326/993] 10.0.0.210 () {62 vars in 1346 bytes} [Wed Apr 12 02:25:53 2023] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-930950318 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:26:07 user devstack@n-api.service[72507]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:26:07 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:26:07 user devstack@n-api.service[72507]: Wed Apr 12 02:26:07 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.0.0.210) !!! Apr 12 02:26:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerBootFromVolumeStableRescueTest-server-1066136151", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "block_device_mapping_v2": [{"boot_index": "0", "source_type": "image", "volume_size": 1, "uuid": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "destination_type": "volume"}], "networks": [{"uuid": "68f69bfd-e205-4f13-92e2-346de9c67dd4"}]}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:26:07 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] validate_networks() for [('68f69bfd-e205-4f13-92e2-346de9c67dd4', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Getting quotas for user fe9217f4c1ff428086140b7fd07194eb and project 6bb951049b464316a04647ce9c52963a. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:26:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:26:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-cf029979-b3f1-41c7-a22e-fa511c316fec tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 378 microversion: 2.87 time: 1.011139 Apr 12 02:26:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 333/994] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:26:07 2023] POST /compute/v2.1/servers => generated 378 bytes in 1014 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:08 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-1a0f00a1-924c-471e-9baf-a9f75344a6d4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1395 microversion: 2.87 time: 0.134332 Apr 12 02:26:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 336/995] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:26:08 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1395 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:26:10 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=3,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:26:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] Creating event network-changed:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:26:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a0fbb50a-5c75-44a7-9945-81db03adccd3 req-7f000382-ebec-49d9-9203-4eea9cc77de4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035776 Apr 12 02:26:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 327/996] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:26:10 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:26:22 user devstack@n-api.service[72507]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=13,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:26:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-b3855ba3-6f64-4da2-aca0-900f6c4e341d req-7f69322c-c744-454f-8f89-91565d3399ac service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039256 Apr 12 02:26:22 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 334/997] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:26:22 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:26:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-d980f70e-056c-413d-a4d0-d98825f525b8 req-4aa66573-b235-40ce-b85e-f98f161b7b6b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046283 Apr 12 02:26:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 337/998] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:26:24 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:26:58 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:59 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:59 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:26:59 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-030250bc-96e9-4ee3-94a6-9b1dc37d2574 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1769 microversion: 2.87 time: 0.330078 Apr 12 02:26:59 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 328/999] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:26:58 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1769 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"rescue": {"rescue_image_ref": "14b68404-a3f6-4c9b-be99-7ca9303c3776"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:27:59 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 59642. Apr 12 02:27:59 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lazy-loading 'flavor' on Instance uuid e9b52057-9ce7-42c0-ae32-1c01cf92ce39 {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:27:59 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-df90a9f5-af84-496a-bd83-fe1795c3a1c6 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39/action" status: 200 len: 29 microversion: 2.87 time: 0.419078 Apr 12 02:27:59 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 335/1000] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:27:59 2023] POST /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39/action => generated 29 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:27:59 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:00 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:00 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:00 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:00 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-264c45a8-4f04-476a-85c2-30051337322d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1775 microversion: 2.87 time: 0.593433 Apr 12 02:28:00 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 338/1001] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:27:59 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1775 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-unplugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] Creating event network-vif-unplugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3808eeff-2b7f-445a-9b82-f70bfe8193ba req-de599b47-fb59-4371-8cef-89b26edd5514 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037083 Apr 12 02:28:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 329/1002] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:28:01 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:28:03 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}, {"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}, {"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}, {"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-unplugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}, {"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:28:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:03 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Creating event network-vif-unplugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:03 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-5f9fb3e5-3df7-4b15-8b08-187f9bb52832 req-095a53d1-b387-4116-972e-0b163ae0c63a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 869 microversion: 2.1 time: 0.034069 Apr 12 02:28:03 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 336/1003] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:28:03 2023] POST /compute/v2.1/os-server-external-events => generated 869 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:28:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:28:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-8f360d83-5016-4900-8598-845930bee0ea req-d1755c81-6d64-4d14-b9e1-b0b611297f29 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035798 Apr 12 02:28:05 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 339/1004] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:28:05 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:28:07 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:28:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:07 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:28:07 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c84f6dba-4d99-42ad-9cd5-f9da6dd7c239 req-f9f8960f-82e6-4794-aa22-3a4538d120ce service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033983 Apr 12 02:28:07 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 330/1005] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:28:07 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=19,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:50 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:28:50 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-8e067153-de53-4f01-85c0-db1874b0e799 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1755 microversion: 2.87 time: 0.175109 Apr 12 02:28:50 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 337/1006] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:28:50 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1755 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-88688fef-091c-4ec3-ac13-3fdd85cc9138 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "POST /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39/action" status: 202 len: 0 microversion: 2.87 time: 0.087045 Apr 12 02:29:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 340/1007] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:29:50 2023] POST /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:29:50 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:29:50 user devstack@n-api.service[72505]: Wed Apr 12 02:29:50 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 (ip 10.0.0.210) !!! Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:29:50 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:29:50 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0d1496cb-c671-4a73-86b3-4c53804b0f2c tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1763 microversion: 2.87 time: 0.189502 Apr 12 02:29:50 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 331/1008] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:29:50 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1763 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-unplugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:51 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:51 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] Creating event network-vif-unplugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:29:51 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-0ab06071-277b-4422-92d0-01d705ba43c5 req-99fcc231-e4b5-4db6-8460-ca37718cf52f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.105735 Apr 12 02:29:51 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 338/1009] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:29:51 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:29:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}, {"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:29:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:29:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:29:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-ad306677-ead1-4385-8468-4b282fd874af req-abfe795e-aed4-4b47-810d-8185348fd7d8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.105911 Apr 12 02:29:53 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 341/1010] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:29:53 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:29:55 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:29:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:29:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:29:55 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:29:55 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:29:55 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-32bd896c-e43f-41b3-96a5-da38bf8e07bf req-f224f23b-1b17-417d-a146-ad765a8b1bb8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033671 Apr 12 02:29:55 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 332/1011] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:29:55 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:30:41 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:30:41 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e44b2c24-a262-42ac-8080-8e2a79e8ae78 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1769 microversion: 2.87 time: 0.174331 Apr 12 02:30:41 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 339/1012] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:30:41 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1769 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:41 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:31:41 user devstack@n-api.service[72506]: Wed Apr 12 02:31:41 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 (ip 10.0.0.210) !!! Apr 12 02:31:41 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:31:41 user devstack@n-api.service[72506]: Wed Apr 12 02:31:41 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 (ip 10.0.0.210) !!! Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:41 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:31:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c0ba466e-7ec0-452d-be7a-6b06e39b7a5b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 204 len: 0 microversion: 2.87 time: 0.361248 Apr 12 02:31:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 342/1013] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:31:41 2023] DELETE /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:31:41 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Apr 12 02:31:41 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:31:41 user devstack@n-api.service[72505]: Wed Apr 12 02:31:41 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 (ip 10.0.0.210) !!! Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:31:41 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:31:41 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f2d914c4-9206-47e8-b14c-74848c7bba58 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 200 len: 1775 microversion: 2.87 time: 0.284019 Apr 12 02:31:41 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 333/1014] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:31:41 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 1775 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-unplugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] Creating event network-vif-unplugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:31:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7e59ab5e-3cc2-47fa-b567-0907f797f522 req-6f679cf1-1b8c-4fb5-b767-e78e4e5de866 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035544 Apr 12 02:31:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 340/1015] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:31:42 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:31:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-deleted", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:31:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] Creating event network-vif-deleted:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:31:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-a020e25c-4bb6-49f0-b8d2-1b795917289f req-3f475eba-ab5e-4611-8675-d5e15a2d1b49 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105224 Apr 12 02:31:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 343/1016] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:31:43 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:31:44 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e9b52057-9ce7-42c0-ae32-1c01cf92ce39", "name": "network-vif-plugged", "status": "completed", "tag": "10e7ad4b-208f-4fee-bdd3-2f9c98e34853"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:31:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:31:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:31:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:31:44 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] Creating event network-vif-plugged:10e7ad4b-208f-4fee-bdd3-2f9c98e34853 for instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 on user Apr 12 02:31:44 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-fda8e530-c366-4452-b713-b66539549ee1 req-5e346d5c-f5f4-45a9-9aa4-c8d3d3a4c3b2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030041 Apr 12 02:31:44 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 334/1017] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:31:44 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: e9b52057-9ce7-42c0-ae32-1c01cf92ce39] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] HTTP exception thrown: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Request method failure captured: Apr 12 02:32:32 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 HTTP/1.1 Apr 12 02:32:32 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:32:32 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:32:32 user devstack@n-api.service[72507]: Connection: close Apr 12 02:32:32 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:32:32 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:32:32 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:32:32 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:32:32 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhQO3Cy6oo1oKlGdxn0HH7yUNPZa8N_0LdNupayZP7ZdchfdGV9ubZHpEM8Pkq9FuWARStsF2VortKtvKDgYnpSxbJROw6-8AXudZPbieyCOfAIjDUHhhn7xvI15eb5JzzAkCq8bWgT-OzMxDO_c53Bz7Vi7syKTNBQcUf8DPmQPCm0 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.87 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Project-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6bb951049b464316a04647ce9c52963a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Tenant-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:32:32 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:32:32 user devstack@n-api.service[72507]: X-User: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:32:32 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:32:32 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:32:32 user devstack@n-api.service[72507]: X-User-Id: fe9217f4c1ff428086140b7fd07194eb Apr 12 02:32:32 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:32:32 user devstack@n-api.service[72507]: method: > Apr 12 02:32:32 user devstack@n-api.service[72507]: exception: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. Apr 12 02:32:32 user devstack@n-api.service[72507]: action_args: {'id': 'e9b52057-9ce7-42c0-ae32-1c01cf92ce39'} Apr 12 02:32:32 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. Apr 12 02:32:32 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Returning 404 to user: Instance e9b52057-9ce7-42c0-ae32-1c01cf92ce39 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:32:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b92aab58-5fe5-4468-b86d-bf709a6d6bfc tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39" status: 404 len: 111 microversion: 2.87 time: 0.042433 Apr 12 02:32:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 341/1018] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:32:32 2023] GET /compute/v2.1/servers/e9b52057-9ce7-42c0-ae32-1c01cf92ce39 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:32:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-29bcce96-3470-42ad-87c3-fa30c301ac3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 204 len: 0 microversion: 2.87 time: 0.132888 Apr 12 02:32:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 344/1019] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:32:32 2023] DELETE /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:32:32 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 43746. Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:32:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:32:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6614a115-c2a0-46db-960f-a1c767ce891d tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 200 len: 1698 microversion: 2.87 time: 0.257078 Apr 12 02:32:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 335/1020] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:32:32 2023] GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 1698 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "name": "network-vif-unplugged", "status": "completed", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] Creating event network-vif-unplugged:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:32:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-44d7d723-1aab-4098-a939-91ac7d8692f1 req-258a6f87-b168-4080-937b-e05d9a696a98 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.116312 Apr 12 02:32:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 342/1021] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:32:32 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:32:33 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "name": "network-vif-deleted", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:32:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:33 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] Creating event network-vif-deleted:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:32:33 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-367f256a-da3d-4959-af4e-b6ed60747dea req-88cfddc2-6b1f-4f74-b9ef-5b9bae1f2f11 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033629 Apr 12 02:32:33 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 345/1022] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:32:33 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:32:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35de4fd0-e7b2-484c-b261-fca54a8103a8", "name": "network-vif-plugged", "status": "completed", "tag": "30f13766-7037-4c15-82c7-4c1a2eb1f889"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:32:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:32:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:32:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:32:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] Creating event network-vif-plugged:30f13766-7037-4c15-82c7-4c1a2eb1f889 for instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 on user Apr 12 02:32:35 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f33f20a5-e545-4ddd-9fca-3fa0fc21dc7d req-24abe6c2-2359-4cf1-a920-f8f6dcb596d1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031523 Apr 12 02:32:35 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 336/1023] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:32:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] AMQP server on 10.0.0.210:5672 is unreachable: Server unexpectedly closed connection. Trying again in 1 seconds.: OSError: Server unexpectedly closed connection Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 35de4fd0-e7b2-484c-b261-fca54a8103a8] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] HTTP exception thrown: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Request method failure captured: Apr 12 02:33:22 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 HTTP/1.1 Apr 12 02:33:22 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:33:22 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:33:22 user devstack@n-api.service[72507]: Connection: close Apr 12 02:33:22 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:33:22 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:33:22 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:33:22 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:33:22 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhQO3Cy6oo1oKlGdxn0HH7yUNPZa8N_0LdNupayZP7ZdchfdGV9ubZHpEM8Pkq9FuWARStsF2VortKtvKDgYnpSxbJROw6-8AXudZPbieyCOfAIjDUHhhn7xvI15eb5JzzAkCq8bWgT-OzMxDO_c53Bz7Vi7syKTNBQcUf8DPmQPCm0 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.87 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Project-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6bb951049b464316a04647ce9c52963a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Tenant-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:33:22 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:33:22 user devstack@n-api.service[72507]: X-User: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:33:22 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:33:22 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:33:22 user devstack@n-api.service[72507]: X-User-Id: fe9217f4c1ff428086140b7fd07194eb Apr 12 02:33:22 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:33:22 user devstack@n-api.service[72507]: method: > Apr 12 02:33:22 user devstack@n-api.service[72507]: exception: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. Apr 12 02:33:22 user devstack@n-api.service[72507]: action_args: {'id': '35de4fd0-e7b2-484c-b261-fca54a8103a8'} Apr 12 02:33:22 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. Apr 12 02:33:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:33:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Returning 404 to user: Instance 35de4fd0-e7b2-484c-b261-fca54a8103a8 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:33:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-adf9ba59-147e-46b4-ad02-a6c2f665096a tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8" status: 404 len: 111 microversion: 2.87 time: 0.043869 Apr 12 02:33:22 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 343/1024] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:33:22 2023] GET /compute/v2.1/servers/35de4fd0-e7b2-484c-b261-fca54a8103a8 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:22 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:33:22 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-17393fbc-380d-4d61-8d82-857af0b94ccd tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 204 len: 0 microversion: 2.87 time: 0.140033 Apr 12 02:33:22 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 346/1025] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:33:22 2023] DELETE /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:33:22 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:33:22 user devstack@n-api.service[72505]: Wed Apr 12 02:33:22 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 (ip 10.0.0.210) !!! Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:33:23 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:33:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-40faea52-15f5-4f48-a287-185a825afc1b tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 200 len: 1776 microversion: 2.87 time: 0.292320 Apr 12 02:33:23 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 337/1026] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:33:22 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 1776 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Apr 12 02:33:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-unplugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:33:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] Creating event network-vif-unplugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:33:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e070d9fe-17ad-4d66-9100-bac3b2c4ee4e req-66540cc8-d26a-401b-be71-d9f59562e0e6 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035949 Apr 12 02:33:23 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 344/1027] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:33:23 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:33:24 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-deleted", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:33:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:24 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] Creating event network-vif-deleted:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:33:24 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4182e4b5-2ca4-47ad-be48-e2840f55b843 req-30790bc1-b4d1-42f6-a566-ee6574e37120 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044780 Apr 12 02:33:24 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 347/1028] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:33:24 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:33:25 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "811c4fb7-d984-461e-9190-4d0e9ec345c2", "name": "network-vif-plugged", "status": "completed", "tag": "217714fd-b1ef-4cf0-8f00-ec948d907b1b"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:33:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:33:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:33:25 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:33:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] Creating event network-vif-plugged:217714fd-b1ef-4cf0-8f00-ec948d907b1b for instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 on user Apr 12 02:33:25 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f749c606-086c-4eef-897f-7eceb2aee74a req-dc37b478-1530-44ba-bd21-4bd6dec9eca8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032525 Apr 12 02:33:25 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 338/1029] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:33:25 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:34:13 user devstack@n-api.service[72507]: Wed Apr 12 02:34:13 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 (ip 10.0.0.210) !!! Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:34:13 user devstack@n-api.service[72507]: Wed Apr 12 02:34:13 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 (ip 10.0.0.210) !!! Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 811c4fb7-d984-461e-9190-4d0e9ec345c2] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 54046. Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] HTTP exception thrown: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Request method failure captured: Apr 12 02:34:13 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 HTTP/1.1 Apr 12 02:34:13 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:34:13 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:34:13 user devstack@n-api.service[72507]: Connection: close Apr 12 02:34:13 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:34:13 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:34:13 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:34:13 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:34:13 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhQO3Cy6oo1oKlGdxn0HH7yUNPZa8N_0LdNupayZP7ZdchfdGV9ubZHpEM8Pkq9FuWARStsF2VortKtvKDgYnpSxbJROw6-8AXudZPbieyCOfAIjDUHhhn7xvI15eb5JzzAkCq8bWgT-OzMxDO_c53Bz7Vi7syKTNBQcUf8DPmQPCm0 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.87 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Project-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Project-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6bb951049b464316a04647ce9c52963a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Tenant: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Tenant-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:34:13 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:34:13 user devstack@n-api.service[72507]: X-User: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:34:13 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:34:13 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:34:13 user devstack@n-api.service[72507]: X-User-Id: fe9217f4c1ff428086140b7fd07194eb Apr 12 02:34:13 user devstack@n-api.service[72507]: X-User-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:34:13 user devstack@n-api.service[72507]: method: > Apr 12 02:34:13 user devstack@n-api.service[72507]: exception: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. Apr 12 02:34:13 user devstack@n-api.service[72507]: action_args: {'id': '811c4fb7-d984-461e-9190-4d0e9ec345c2'} Apr 12 02:34:13 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. Apr 12 02:34:13 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Returning 404 to user: Instance 811c4fb7-d984-461e-9190-4d0e9ec345c2 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1c65986c-687a-4b73-93fc-02df7ca57512 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2" status: 404 len: 111 microversion: 2.87 time: 0.044453 Apr 12 02:34:13 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 345/1030] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:34:13 2023] GET /compute/v2.1/servers/811c4fb7-d984-461e-9190-4d0e9ec345c2 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:34:13 user devstack@n-api.service[72506]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] AMQP server on 10.0.0.210:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:34:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bc385d1a-eb93-44af-a52e-15674c5fac46 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 204 len: 0 microversion: 2.87 time: 0.119288 Apr 12 02:34:13 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 348/1031] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:34:13 2023] DELETE /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:34:13 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fe9217f4c1ff428086140b7fd07194eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6bb951049b464316a04647ce9c52963a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:34:13 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0dc72b72-2860-409f-9300-e6322b221c3e tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 200 len: 1697 microversion: 2.87 time: 0.183357 Apr 12 02:34:13 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 339/1032] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:34:13 2023] GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 1697 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-unplugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:13 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] Creating event network-vif-unplugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:13 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-39b40032-1f97-4d15-b656-759309c1ec27 req-51c7fb8a-f658-4bcb-90cb-d3585887d4e0 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030992 Apr 12 02:34:13 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 346/1033] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:34:13 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:34:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}, {"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}, {"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}, {"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-plugged", "status": "completed", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}, {"server_uuid": "3036c0b5-7987-4ff5-9edf-ee7397cf7beb", "name": "network-vif-deleted", "tag": "162c49eb-7ebe-4a3d-aac4-10060fde3edf"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:34:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:34:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:34:16 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Creating event network-vif-plugged:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] Creating event network-vif-deleted:162c49eb-7ebe-4a3d-aac4-10060fde3edf for instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb on user Apr 12 02:34:16 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-4a63506d-ec8f-46d5-80f2-bdc13d8e7086 req-1cb236b3-fb51-4b3c-90ed-5bce83e32e65 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 867 microversion: 2.1 time: 0.035858 Apr 12 02:34:16 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 349/1034] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:34:16 2023] POST /compute/v2.1/os-server-external-events => generated 867 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] [instance: 3036c0b5-7987-4ff5-9edf-ee7397cf7beb] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:35:03 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:03 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:04 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] HTTP exception thrown: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. Apr 12 02:35:04 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Request method failure captured: Apr 12 02:35:04 user devstack@n-api.service[72505]: request: GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb HTTP/1.1 Apr 12 02:35:04 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:35:04 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:35:04 user devstack@n-api.service[72505]: Connection: close Apr 12 02:35:04 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:35:04 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:35:04 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:35:04 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:35:04 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhQO3Cy6oo1oKlGdxn0HH7yUNPZa8N_0LdNupayZP7ZdchfdGV9ubZHpEM8Pkq9FuWARStsF2VortKtvKDgYnpSxbJROw6-8AXudZPbieyCOfAIjDUHhhn7xvI15eb5JzzAkCq8bWgT-OzMxDO_c53Bz7Vi7syKTNBQcUf8DPmQPCm0 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.87 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Project-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Project-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6bb951049b464316a04647ce9c52963a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6bb951049b464316a04647ce9c52963a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Tenant: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Tenant-Id: 6bb951049b464316a04647ce9c52963a Apr 12 02:35:04 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577 Apr 12 02:35:04 user devstack@n-api.service[72505]: X-User: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:35:04 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:35:04 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:35:04 user devstack@n-api.service[72505]: X-User-Id: fe9217f4c1ff428086140b7fd07194eb Apr 12 02:35:04 user devstack@n-api.service[72505]: X-User-Name: tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member Apr 12 02:35:04 user devstack@n-api.service[72505]: method: > Apr 12 02:35:04 user devstack@n-api.service[72505]: exception: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. Apr 12 02:35:04 user devstack@n-api.service[72505]: action_args: {'id': '3036c0b5-7987-4ff5-9edf-ee7397cf7beb'} Apr 12 02:35:04 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. Apr 12 02:35:04 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:35:04 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] Returning 404 to user: Instance 3036c0b5-7987-4ff5-9edf-ee7397cf7beb could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:35:04 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-a94e8b07-3f29-47fa-8d25-7d28de9139e4 tempest-ServerBootFromVolumeStableRescueTest-1070279577 tempest-ServerBootFromVolumeStableRescueTest-1070279577-project-member] 10.0.0.210 "GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb" status: 404 len: 111 microversion: 2.87 time: 0.037816 Apr 12 02:35:04 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 340/1035] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:35:03 2023] GET /compute/v2.1/servers/3036c0b5-7987-4ff5-9edf-ee7397cf7beb => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:35:14 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:14 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:14 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-8a61b84b-a991-4f81-8925-d06aff159b2b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: .version_select at 0x7fb202fb80d0>, body: {"keypair": {"name": "tempest-keypair-2136830748", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJtoSQHC4yFKsSO0sQxkGPQvZnabNgsGvZ+PAGIKtWK5Je1BVouvWpb7OzRABncPbZnXWYtel/GG2A5dh7y+j1UOv9Y0st7v2BuEsWOkf4zfAMZhgkbwTSXP2ko7gvb2cQ=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:35:14 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-8a61b84b-a991-4f81-8925-d06aff159b2b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:35:14 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-8a61b84b-a991-4f81-8925-d06aff159b2b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for user b14e24aaf4e94584a824ff4886e89db6 and project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:35:14 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-8a61b84b-a991-4f81-8925-d06aff159b2b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.163618 Apr 12 02:35:14 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 347/1036] 10.0.0.210 () {66 vars in 1277 bytes} [Wed Apr 12 02:35:14 2023] POST /compute/v2.1/os-keypairs => generated 401 bytes in 165 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Apr 12 02:35:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:16 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:16 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1616847054", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "ab5d72ac-f30c-4d4d-ad2c-fb6b6f871e09"}], "security_groups": [{"name": "tempest-securitygroup--1326979778"}], "key_name": "tempest-keypair-2136830748", "user_data": "}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:35:16 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 37538. Apr 12 02:35:16 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] validate_networks() for [('ab5d72ac-f30c-4d4d-ad2c-fb6b6f871e09', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for user b14e24aaf4e94584a824ff4886e89db6 and project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:35:17 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:35:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cedfcceb-e8f1-48ab-82ac-d3071e6f2872 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 1.068716 Apr 12 02:35:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 350/1037] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:35:16 2023] POST /compute/v2.1/servers => generated 400 bytes in 1071 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:35:17 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 47168. Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:35:17 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:35:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1b0f26e9-6b3d-4689-97dc-f7e1fc2564bf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1276 microversion: 2.20 time: 0.086466 Apr 12 02:35:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 341/1038] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:35:17 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1276 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:35:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:35:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:19 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] Creating event network-changed:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:35:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-17f5ba63-21ce-40b6-b8d8-b753f428eff8 req-cb7f9fc9-53a4-4d9b-a6bd-59271e93f2a7 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029351 Apr 12 02:35:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 348/1039] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:35:19 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:35:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:35:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:21 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:35:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-b0591b10-dfa3-4956-b9b4-af4b988cf6a2 req-12a9968c-2913-496c-86fe-186c6c9e4a46 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101647 Apr 12 02:35:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 351/1040] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:35:21 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:35:23 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:35:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:35:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:35:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:35:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:35:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7ade8059-f0a2-449d-b076-b8a0f857d23f req-10bd8420-c122-43b2-b613-07f52903f8a2 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106954 Apr 12 02:35:23 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 342/1041] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:35:23 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=13,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:36:07 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:36:07 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-bfc7c1cd-5934-4ad0-ab1a-18f1082a7767 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1588 microversion: 2.20 time: 0.270149 Apr 12 02:36:07 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 349/1042] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:36:07 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1588 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:07 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:37:07 user devstack@n-api.service[72506]: Wed Apr 12 02:37:07 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-interface (ip 10.0.0.210) !!! Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:07 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:07 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7d8334e7-56df-4c45-87be-1fcc83d27fc1 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-interface" status: 200 len: 282 microversion: 2.20 time: 0.116833 Apr 12 02:37:07 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 352/1043] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 02:37:07 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-interface => generated 282 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:08 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:37:08 user devstack@n-api.service[72505]: Wed Apr 12 02:37:08 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:08 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7538c15c-4af4-4500-a1e1-b8953de363a5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/ips" status: 200 len: 110 microversion: 2.20 time: 0.047507 Apr 12 02:37:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 350/1044] 10.0.0.210 () {64 vars in 1366 bytes} [Wed Apr 12 02:37:08 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/ips => generated 110 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:37:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] Creating event network-changed:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:37:08 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-0052c9c1-3aa5-429a-b0fc-c66193ef263b req-e0a494a9-1a8a-4645-899e-5787dc7a0205 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.125571 Apr 12 02:37:08 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 343/1045] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:37:08 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:10 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lazy-loading 'flavor' on Instance uuid 322ac98d-d990-4841-b98c-74a53382efd6 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:37:11 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e844875c-2c58-44b6-a201-79d22a31c4b4 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/action" status: 202 len: 0 microversion: 2.20 time: 0.908901 Apr 12 02:37:11 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 353/1046] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:37:10 2023] POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/action => generated 0 bytes in 911 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:37:11 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:37:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-05315493-8d6b-4bb1-b68a-03c940a99ed6 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1712 microversion: 2.20 time: 0.191402 Apr 12 02:37:11 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 351/1047] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:37:11 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1712 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:37:11 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-unplugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:37:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:11 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:11 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] Creating event network-vif-unplugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:37:11 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-93f54781-de9f-4498-8819-2a371ac9473e req-58cd0d1a-a607-4baa-9f84-1fddd1c5ef3c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032269 Apr 12 02:37:11 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 344/1048] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:37:11 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:37:13 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:37:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:13 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:37:13 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-1631e469-0ffc-45ed-91e4-c384a33a7908 req-171538c2-3554-4bda-8c6f-4b0e78d35df9 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037772 Apr 12 02:37:13 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 354/1049] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:37:13 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:37:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:37:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:37:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:37:16 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:37:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] Creating event network-changed:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:37:16 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-156935fd-2292-404b-a7e8-a06378c7321b req-34976eab-7fb6-43a2-841c-c569dd9dfc17 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.028058 Apr 12 02:37:16 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 352/1050] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:37:15 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:38:01 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:38:01 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-9281877b-6b59-44d9-84f4-cf4ce38d55ae tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1653 microversion: 2.20 time: 0.253295 Apr 12 02:38:01 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 345/1051] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:38:01 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1653 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:39:01 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:39:01 user devstack@n-api.service[72506]: Wed Apr 12 02:39:01 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments (ip 10.0.0.210) !!! Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d7e20126-acbc-421c-9437-88099560832f"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:01 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:02 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-10e82ee7-e74b-4750-8010-8ac682bd6c5a tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.503515 Apr 12 02:39:02 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 355/1052] 10.0.0.210 () {66 vars in 1441 bytes} [Wed Apr 12 02:39:01 2023] POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments => generated 188 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=12 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:02 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=72507) unshelve /opt/stack/nova/nova/compute/api.py:4596}} Apr 12 02:39:02 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-6b31e27e-7e71-40b9-bc1e-421f5b7e578b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/action" status: 202 len: 0 microversion: 2.20 time: 0.175965 Apr 12 02:39:02 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 353/1053] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:39:02 2023] POST /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/action => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:02 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:39:02 user devstack@n-api.service[72505]: Wed Apr 12 02:39:02 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 (ip 10.0.0.210) !!! Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:02 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:02 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3741142f-1fcf-4f26-addc-2fdabddd5793 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1739 microversion: 2.20 time: 0.182839 Apr 12 02:39:02 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 346/1054] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:39:02 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1739 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:39:09 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:39:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:09 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] Creating event network-changed:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:39:09 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-462d983a-76fa-40c0-8197-bc86efc2b0c1 req-d1e94150-87f0-4cd6-888c-2846a53d3b7e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033058 Apr 12 02:39:09 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 356/1055] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:39:09 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:39:17 user devstack@n-api.service[72507]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] AMQP server on 10.0.0.210:5672 is unreachable: Server unexpectedly closed connection. Trying again in 1 seconds.: OSError: Server unexpectedly closed connection Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=10,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:17 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Apr 12 02:39:17 user devstack@n-api.service[72507]: Wed Apr 12 02:39:17 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:39:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-03453cfe-bd04-4417-a945-a64e5b694f3e req-a27bb279-ca99-4976-8039-0d40f4e86a03 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042975 Apr 12 02:39:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 354/1056] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:39:17 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:39:19 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:19 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:39:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-1ea37ad1-9e98-45d8-b0f2-63981971cfcc req-e4497453-68f8-4bfe-9ba3-445d44559462 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042666 Apr 12 02:39:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 347/1057] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:39:19 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:52 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:39:52 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-7e3c17d0-3205-4e4f-8cf3-4bf2f0362784 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1784 microversion: 2.20 time: 0.166956 Apr 12 02:39:52 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 357/1058] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:39:52 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1784 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:40:52 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:40:52 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 38978. Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-cfa720e6-c278-4ca8-a7ba-4d6336402240 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments/d7e20126-acbc-421c-9437-88099560832f" status: 200 len: 194 microversion: 2.20 time: 0.106818 Apr 12 02:40:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 355/1059] 10.0.0.210 () {64 vars in 1531 bytes} [Wed Apr 12 02:40:52 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments/d7e20126-acbc-421c-9437-88099560832f => generated 194 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 38994. Apr 12 02:40:53 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lazy-loading 'services' on Instance uuid 322ac98d-d990-4841-b98c-74a53382efd6 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:40:53 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-479dbae3-de54-44c8-ac9d-b75d1a3d4267 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments/d7e20126-acbc-421c-9437-88099560832f" status: 202 len: 0 microversion: 2.20 time: 0.232954 Apr 12 02:40:53 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 348/1060] 10.0.0.210 () {64 vars in 1534 bytes} [Wed Apr 12 02:40:53 2023] DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments/d7e20126-acbc-421c-9437-88099560832f => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:40:53 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:40:53 user devstack@n-api.service[72506]: Wed Apr 12 02:40:53 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments (ip 10.0.0.210) !!! Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:40:53 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:40:53 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6149a0d7-0a39-43f0-b291-9dd3e68c50f2 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057746 Apr 12 02:40:53 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 358/1061] 10.0.0.210 () {64 vars in 1420 bytes} [Wed Apr 12 02:40:53 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-143e5eb1-9a33-4625-ab4e-50f487d84cba tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.053363 Apr 12 02:41:43 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 356/1062] 10.0.0.210 () {64 vars in 1420 bytes} [Wed Apr 12 02:41:43 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:41:43 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-8615e788-a47a-481c-a559-d5d6b5a4bbaf tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 204 len: 0 microversion: 2.20 time: 0.139797 Apr 12 02:41:43 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 349/1063] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:41:43 2023] DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:41:43 user devstack@n-api.service[72506]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:41:43 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:41:43 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-c1fe83b4-c0f3-46c5-a6c1-2a41f9636652 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 200 len: 1712 microversion: 2.20 time: 0.254196 Apr 12 02:41:43 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 359/1064] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:41:43 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 1712 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:41:44 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-unplugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:41:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:44 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] Creating event network-vif-unplugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:41:44 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-eb500bef-47cc-4b43-8627-f30ece055494 req-3b9502a1-2917-4319-a39e-f703abfcdfb5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.113795 Apr 12 02:41:44 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 357/1065] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:41:43 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:41:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-deleted", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:41:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:45 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:45 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] Creating event network-vif-deleted:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:41:45 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8f5a389c-496b-4cad-b000-177b66a0e40a req-3fe6eb9e-0dda-4bcb-a711-aaf3f348d4a1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034137 Apr 12 02:41:45 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 350/1066] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:41:45 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:41:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "322ac98d-d990-4841-b98c-74a53382efd6", "name": "network-vif-plugged", "status": "completed", "tag": "5b85d915-02dc-4e6b-9b28-e873d9b61c7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:41:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:41:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:41:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:41:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] Creating event network-vif-plugged:5b85d915-02dc-4e6b-9b28-e873d9b61c7d for instance 322ac98d-d990-4841-b98c-74a53382efd6 on user Apr 12 02:41:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-2d72afb0-7be5-4044-8865-99d751f2bbb7 req-03fedcaa-ab64-4786-8a48-d4e6b9539af1 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032681 Apr 12 02:41:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 360/1067] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:41:46 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:34 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Apr 12 02:42:34 user devstack@n-api.service[72507]: Wed Apr 12 02:42:34 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 (ip 10.0.0.210) !!! Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:42:34 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 HTTP/1.1 Apr 12 02:42:34 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:42:34 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:42:34 user devstack@n-api.service[72507]: Connection: close Apr 12 02:42:34 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:42:34 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:42:34 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:42:34 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:42:34 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:42:34 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:42:34 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:42:34 user devstack@n-api.service[72507]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:42:34 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:42:34 user devstack@n-api.service[72507]: method: > Apr 12 02:42:34 user devstack@n-api.service[72507]: exception: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:42:34 user devstack@n-api.service[72507]: action_args: {'id': '322ac98d-d990-4841-b98c-74a53382efd6'} Apr 12 02:42:34 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:42:34 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:42:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:42:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e7edfe14-61f6-4ef3-b929-90312680e593 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 404 len: 111 microversion: 2.20 time: 0.050165 Apr 12 02:42:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 358/1068] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:42:34 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:42:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:34 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-6b525266-8d3c-43fe-955e-3c16be067bec tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '.version_select at 0x7fb2035e6710>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:42:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-6b525266-8d3c-43fe-955e-3c16be067bec tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2136830748" status: 204 len: 0 microversion: 2.20 time: 0.044985 Apr 12 02:42:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 351/1069] 10.0.0.210 () {64 vars in 1339 bytes} [Wed Apr 12 02:42:34 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2136830748 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:42:35 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:35 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=8 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:42:35 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 47780. Apr 12 02:42:35 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-8ae78b07-a274-447d-a64d-f6c9910b43c3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: .version_select at 0x7fb203150940>, body: {"keypair": {"name": "tempest-keypair-1235094926", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBMQIrHz8sj+8MEnNCUkf4pCjv1uMpLhOgu3rcw8mO7hzQH8hWLqElQtlss11G+BYSe8GnwaJ5uvIv/A3TdUM5Ufms9xwOH+0sUifvltzb/i72wXhv4++3eQoDWz+m9bsQ=="}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:42:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-8ae78b07-a274-447d-a64d-f6c9910b43c3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:42:35 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-8ae78b07-a274-447d-a64d-f6c9910b43c3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for user b14e24aaf4e94584a824ff4886e89db6 and project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'key_pairs'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:42:35 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-8ae78b07-a274-447d-a64d-f6c9910b43c3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.057765 Apr 12 02:42:35 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 361/1070] 10.0.0.210 () {66 vars in 1277 bytes} [Wed Apr 12 02:42:35 2023] POST /compute/v2.1/os-keypairs => generated 401 bytes in 60 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Apr 12 02:42:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1452164160", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "adminPass": "***", "networks": [{"uuid": "ab5d72ac-f30c-4d4d-ad2c-fb6b6f871e09"}], "security_groups": [{"name": "tempest-securitygroup--991468991"}], "key_name": "tempest-keypair-1235094926", "user_data": "}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.network.neutron [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] validate_networks() for [('ab5d72ac-f30c-4d4d-ad2c-fb6b6f871e09', None, None, None, None, None)] {{(pid=72507) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Flavor limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Image limits 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Flavor pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Image pref 0:0:0 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.virt.hardware [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72507) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Getting quotas for user b14e24aaf4e94584a824ff4886e89db6 and project ef2245b2b35848928e8e53439a1d9cb6. Resources: {'cores', 'instances', 'ram'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Going to run 1 instances... {{(pid=72507) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:42:37 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72507) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:42:37 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-59033a05-2f6b-4694-805a-fbfe5f20cb18 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 399 microversion: 2.20 time: 0.651997 Apr 12 02:42:37 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 359/1071] 10.0.0.210 () {66 vars in 1265 bytes} [Wed Apr 12 02:42:36 2023] POST /compute/v2.1/servers => generated 399 bytes in 654 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Apr 12 02:42:37 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:42:37 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:42:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:42:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:42:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:42:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5f2a533a-7cbc-4818-bb89-5550c4217da5 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1276 microversion: 2.20 time: 0.161149 Apr 12 02:42:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 352/1072] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:42:37 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1276 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:42:39 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:42:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:40 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] Creating event network-changed:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:42:40 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-42a80218-f2c7-4648-b27e-cd844e36064e req-c65d4d46-9594-4e02-9365-84da0ec93b72 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083979 Apr 12 02:42:40 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 362/1073] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:42:39 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:42:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:42:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:42 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:42:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7a95777f-c9ac-4562-b7fe-0b12f170cb41 req-762030ce-eb0f-44c3-8912-257936ef092d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029054 Apr 12 02:42:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 360/1074] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:42:42 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:42:44 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:42:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:42:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:42:44 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:42:44 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:42:44 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-2cae7110-0a62-4765-9cb1-8ec79892e379 req-e5b0cf62-a58d-4c33-88d1-6793a16cff44 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034313 Apr 12 02:42:44 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 353/1075] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:42:44 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=13,nova_api:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:43:27 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:43:27 user devstack@n-api.service[72506]: Wed Apr 12 02:43:27 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a (ip 10.0.0.210) !!! Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:43:27 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:43:28 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:43:28 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:43:28 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:43:28 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-025aeab1-3e90-4772-9500-ea495562adcc tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1587 microversion: 2.20 time: 0.189932 Apr 12 02:43:28 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 363/1076] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:43:27 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1587 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=19,nova_api:INSERT=3 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:28 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:28 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-956360c1-1cff-4696-93a0-26be64918a8d tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-interface" status: 200 len: 282 microversion: 2.20 time: 0.184055 Apr 12 02:44:28 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 361/1077] 10.0.0.210 () {64 vars in 1393 bytes} [Wed Apr 12 02:44:28 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-interface => generated 282 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:29 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:44:29 user devstack@n-api.service[72505]: Wed Apr 12 02:44:29 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=9,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:29 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:29 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-bb723dbc-6ea8-4569-a1f3-901bb6d50982 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/ips" status: 200 len: 110 microversion: 2.20 time: 0.059048 Apr 12 02:44:29 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 364/1078] 10.0.0.210 () {64 vars in 1366 bytes} [Wed Apr 12 02:44:29 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/ips => generated 110 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:29 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] Creating event network-changed:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:44:29 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-3e129a4f-100f-48d3-84cf-a9a610e237c9 req-724bc48b-7d3c-4d44-a219-d0aea2da0acf service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.142143 Apr 12 02:44:29 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 354/1079] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:44:29 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:30 user devstack@n-api.service[72507]: DEBUG nova.objects.instance [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lazy-loading 'flavor' on Instance uuid 7811c126-63b5-48c7-854b-14b162f2ad7a {{(pid=72507) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:44:31 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-1bccc319-7b9a-4152-81c0-6da421b13d3e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/action" status: 202 len: 0 microversion: 2.20 time: 0.951830 Apr 12 02:44:31 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 362/1080] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:44:30 2023] POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/action => generated 0 bytes in 954 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:44:31 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:44:31 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b1827d7a-a540-4d78-b32f-971150b222eb tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1711 microversion: 2.20 time: 0.280177 Apr 12 02:44:31 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 365/1081] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:44:31 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1711 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:44:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-unplugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:44:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] Creating event network-vif-unplugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:44:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7d62b56e-c80d-4f89-a475-def1537bd77a req-b12c264d-04fc-46a2-acae-676b9167c910 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032573 Apr 12 02:44:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 355/1082] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:44:31 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:44:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:44:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:44:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-1bbf81fb-e795-4f6e-8d40-b997a81556b1 req-d6bf1ce0-29fb-40d9-910d-e243ec2eba8f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033817 Apr 12 02:44:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 363/1083] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:44:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:44:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:44:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:44:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:44:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:44:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] Creating event network-changed:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:44:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-58a0ca7c-78de-4a81-adce-3d8486b54856 req-2d4ca3f9-9030-410c-9945-9341c91bdf50 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.027730 Apr 12 02:44:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 366/1084] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:44:36 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:45:22 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Apr 12 02:45:22 user devstack@n-api.service[72505]: Wed Apr 12 02:45:22 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a (ip 10.0.0.210) !!! Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:45:22 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:45:22 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4a760aef-8828-4a3a-ba54-f2ca6fa1973c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1652 microversion: 2.20 time: 0.474246 Apr 12 02:45:22 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 356/1085] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:45:22 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1652 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:46:22 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:46:22 user devstack@n-api.service[72507]: Wed Apr 12 02:46:22 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments (ip 10.0.0.210) !!! Apr 12 02:46:22 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:46:22 user devstack@n-api.service[72507]: Wed Apr 12 02:46:22 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments (ip 10.0.0.210) !!! Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=21,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "83b964bc-eea5-4c1b-9d41-ccaf18367cd5"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-12ca299c-3dec-4c1b-b810-949418a899cd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.471122 Apr 12 02:46:23 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 364/1086] 10.0.0.210 () {66 vars in 1441 bytes} [Wed Apr 12 02:46:22 2023] POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments => generated 188 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=12 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-19726df5-e8b7-446c-a270-ea4a5fefdd6b tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments/83b964bc-eea5-4c1b-9d41-ccaf18367cd5" status: 202 len: 0 microversion: 2.20 time: 0.233490 Apr 12 02:46:23 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 367/1087] 10.0.0.210 () {64 vars in 1534 bytes} [Wed Apr 12 02:46:23 2023] DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments/83b964bc-eea5-4c1b-9d41-ccaf18367cd5 => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=72505) unshelve /opt/stack/nova/nova/compute/api.py:4596}} Apr 12 02:46:23 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-4ab4c41a-680c-4257-b402-0373a8062ecd tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/action" status: 202 len: 0 microversion: 2.20 time: 0.260600 Apr 12 02:46:23 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 357/1088] 10.0.0.210 () {66 vars in 1396 bytes} [Wed Apr 12 02:46:23 2023] POST /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/action => generated 0 bytes in 266 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:23 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:24 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:46:24 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:46:24 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:46:24 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f0a57bec-c058-4c24-8f9d-ab347bde53c8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1660 microversion: 2.20 time: 0.302967 Apr 12 02:46:24 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 365/1089] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:46:23 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1660 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:46:26 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:46:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:26 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] Creating event network-changed:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:46:26 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-394b5d14-1481-4479-88f9-32b40955fd86 req-3b6d40e5-dd3e-4e97-8d3a-38071620fbe4 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030453 Apr 12 02:46:26 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 368/1090] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:46:26 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:46:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:46:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:46:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-c8024e45-d337-45b3-897c-ffc4fa2d945c req-3c8df1be-27ba-41e0-8502-955da306d1ea service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044740 Apr 12 02:46:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 358/1091] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:46:32 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=21,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:46:34 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:46:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:46:34 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-fff13747-e972-4988-a36a-36c8c10e4614 req-8b1a3719-6dd6-4246-9118-099e39ab0cd5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044160 Apr 12 02:46:34 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 366/1092] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:46:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:47:14 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Apr 12 02:47:14 user devstack@n-api.service[72506]: Wed Apr 12 02:47:14 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a (ip 10.0.0.210) !!! Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:47:14 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:47:14 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-45bd91ad-360c-4b79-9d8d-fe74e67c82d8 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1705 microversion: 2.20 time: 0.249604 Apr 12 02:47:14 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 369/1093] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:47:14 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1705 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=8,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:14 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ed44006c-e21d-4ed6-b2b9-d0e89ad15f61 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.068102 Apr 12 02:48:14 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 359/1094] 10.0.0.210 () {64 vars in 1420 bytes} [Wed Apr 12 02:48:14 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:14 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Going to try to terminate instance {{(pid=72507) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:48:14 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-ed778c9c-ca58-424e-84ae-afd811c1850e tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 204 len: 0 microversion: 2.20 time: 0.130236 Apr 12 02:48:14 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 367/1095] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:48:14 2023] DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:14 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:15 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:48:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-unplugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:48:15 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:48:15 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b14e24aaf4e94584a824ff4886e89db6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ef2245b2b35848928e8e53439a1d9cb6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:48:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e4752d09-510d-47a4-a07d-6d45063c9618 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 200 len: 1711 microversion: 2.20 time: 0.417710 Apr 12 02:48:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 370/1096] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:48:14 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 1711 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Apr 12 02:48:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:15 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] Creating event network-vif-unplugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:48:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-4127c731-dc37-47ea-9e0f-e1fd1b68a1a3 req-5abc9e66-1d80-4151-b74a-f937ab927d5f service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.095450 Apr 12 02:48:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 360/1097] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:48:15 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:48:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-plugged", "status": "completed", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}, {"server_uuid": "7811c126-63b5-48c7-854b-14b162f2ad7a", "name": "network-vif-deleted", "tag": "da4f79fc-e83f-4c90-94a4-240d06066414"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:48:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:48:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:48:17 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:48:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Creating event network-vif-plugged:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:48:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] Creating event network-vif-deleted:da4f79fc-e83f-4c90-94a4-240d06066414 for instance 7811c126-63b5-48c7-854b-14b162f2ad7a on user Apr 12 02:48:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-968303d2-3326-425b-9705-0e7d56baf529 req-baef8dc7-aaf0-4401-9b2d-e5f385e18c5a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.032422 Apr 12 02:48:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 368/1098] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:48:17 2023] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:49:05 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a HTTP/1.1 Apr 12 02:49:05 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:49:05 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:49:05 user devstack@n-api.service[72506]: Connection: close Apr 12 02:49:05 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:49:05 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:49:05 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:49:05 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:49:05 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:05 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:49:05 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:49:05 user devstack@n-api.service[72506]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:49:05 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:05 user devstack@n-api.service[72506]: method: > Apr 12 02:49:05 user devstack@n-api.service[72506]: exception: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:05 user devstack@n-api.service[72506]: action_args: {'id': '7811c126-63b5-48c7-854b-14b162f2ad7a'} Apr 12 02:49:05 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:05 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:05 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:05 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f788c78d-b2b8-4a2b-8969-d8c73f4f932c tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 404 len: 111 microversion: 2.20 time: 0.046426 Apr 12 02:49:05 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 371/1099] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:49:05 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:49:05 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: Server unexpectedly closed connection. Trying again in 1 seconds.: OSError: Server unexpectedly closed connection Apr 12 02:49:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:05 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:05 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Apr 12 02:49:05 user devstack@n-api.service[72505]: Wed Apr 12 02:49:05 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-1235094926 (ip 10.0.0.210) !!! Apr 12 02:49:05 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-f2c27add-5028-4a59-a5b0-9dfd04f5e24f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '.version_select at 0x7fb202e0de10>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:05 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-f2c27add-5028-4a59-a5b0-9dfd04f5e24f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1235094926" status: 204 len: 0 microversion: 2.20 time: 0.039368 Apr 12 02:49:05 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 361/1100] 10.0.0.210 () {64 vars in 1339 bytes} [Wed Apr 12 02:49:05 2023] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1235094926 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:49:08 user devstack@n-api.service[72507]: request: DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a HTTP/1.1 Apr 12 02:49:08 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:49:08 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:49:08 user devstack@n-api.service[72507]: Connection: close Apr 12 02:49:08 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:49:08 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:49:08 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:49:08 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:49:08 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:08 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:49:08 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:49:08 user devstack@n-api.service[72507]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:49:08 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:08 user devstack@n-api.service[72507]: method: > Apr 12 02:49:08 user devstack@n-api.service[72507]: exception: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72507]: action_args: {'id': '7811c126-63b5-48c7-854b-14b162f2ad7a'} Apr 12 02:49:08 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:08 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-ccbe2311-9c78-4ef7-8f1c-42d229486bf3 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 404 len: 111 microversion: 2.20 time: 0.044508 Apr 12 02:49:08 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 369/1101] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:49:08 2023] DELETE /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 7811c126-63b5-48c7-854b-14b162f2ad7a] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:08 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 42670. Apr 12 02:49:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:49:08 user devstack@n-api.service[72506]: request: GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a HTTP/1.1 Apr 12 02:49:08 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:49:08 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:49:08 user devstack@n-api.service[72506]: Connection: close Apr 12 02:49:08 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:49:08 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:49:08 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:49:08 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:49:08 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:08 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:49:08 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:49:08 user devstack@n-api.service[72506]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:49:08 user devstack@n-api.service[72506]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:08 user devstack@n-api.service[72506]: method: > Apr 12 02:49:08 user devstack@n-api.service[72506]: exception: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72506]: action_args: {'id': '7811c126-63b5-48c7-854b-14b162f2ad7a'} Apr 12 02:49:08 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. Apr 12 02:49:08 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:08 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 7811c126-63b5-48c7-854b-14b162f2ad7a could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:08 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6242babf-9ff0-4841-9fb4-ff65f90c9446 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a" status: 404 len: 111 microversion: 2.20 time: 0.052217 Apr 12 02:49:08 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 372/1102] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:49:08 2023] GET /compute/v2.1/servers/7811c126-63b5-48c7-854b-14b162f2ad7a => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:10 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 42684. Apr 12 02:49:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:49:10 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 HTTP/1.1 Apr 12 02:49:10 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:49:10 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:49:10 user devstack@n-api.service[72505]: Connection: close Apr 12 02:49:10 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:49:10 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:49:10 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:49:10 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:49:10 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:10 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:49:10 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:49:10 user devstack@n-api.service[72505]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:49:10 user devstack@n-api.service[72505]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:10 user devstack@n-api.service[72505]: method: > Apr 12 02:49:10 user devstack@n-api.service[72505]: exception: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:10 user devstack@n-api.service[72505]: action_args: {'id': '322ac98d-d990-4841-b98c-74a53382efd6'} Apr 12 02:49:10 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1097, in delete Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 891, in _delete Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:10 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:10 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:10 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-ee04f8c5-19a3-43f2-9200-32156cfeba7f tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 404 len: 111 microversion: 2.20 time: 0.034234 Apr 12 02:49:10 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 362/1103] 10.0.0.210 () {64 vars in 1357 bytes} [Wed Apr 12 02:49:10 2023] DELETE /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] [instance: 322ac98d-d990-4841-b98c-74a53382efd6] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] HTTP exception thrown: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Request method failure captured: Apr 12 02:49:11 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 HTTP/1.1 Apr 12 02:49:11 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:49:11 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:49:11 user devstack@n-api.service[72507]: Connection: close Apr 12 02:49:11 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:49:11 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:49:11 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:49:11 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:49:11 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhjid9lLrvbNoiGoUn93cysUUibsWCNcsFc4QFehyIDpEwwlgKTtL2USqa1VUd-rcpxPb-QEeDBy3HHN1jx3-ypuq5hMdHONM6kgIDsukgNlE_SpE2m47P38TdCUGVr4dkpu6Pe6gih1uaJP077iSLBu1u0X_3Cv72m-hzQKOuJzld4 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Openstack-Nova-Api-Version: 2.20 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Project-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/ef2245b2b35848928e8e53439a1d9cb6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Tenant: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Tenant-Id: ef2245b2b35848928e8e53439a1d9cb6 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-817837534 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-User: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:11 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:49:11 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:49:11 user devstack@n-api.service[72507]: X-User-Id: b14e24aaf4e94584a824ff4886e89db6 Apr 12 02:49:11 user devstack@n-api.service[72507]: X-User-Name: tempest-AttachVolumeShelveTestJSON-817837534-project-member Apr 12 02:49:11 user devstack@n-api.service[72507]: method: > Apr 12 02:49:11 user devstack@n-api.service[72507]: exception: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:11 user devstack@n-api.service[72507]: action_args: {'id': '322ac98d-d990-4841-b98c-74a53382efd6'} Apr 12 02:49:11 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. Apr 12 02:49:11 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:11 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] Returning 404 to user: Instance 322ac98d-d990-4841-b98c-74a53382efd6 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:11 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-b6a11a44-4365-47d3-bb0c-f1cde05d7823 tempest-AttachVolumeShelveTestJSON-817837534 tempest-AttachVolumeShelveTestJSON-817837534-project-member] 10.0.0.210 "GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6" status: 404 len: 111 microversion: 2.20 time: 0.033137 Apr 12 02:49:11 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 370/1104] 10.0.0.210 () {64 vars in 1354 bytes} [Wed Apr 12 02:49:10 2023] GET /compute/v2.1/servers/322ac98d-d990-4841-b98c-74a53382efd6 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Apr 12 02:49:15 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-d1c8ef5f-4f3e-4c74-8f81-26ba680fc4ff tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Action: 'create', calling method: .version_select at 0x7fb202fd7250>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1092306341"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:15 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-d1c8ef5f-4f3e-4c74-8f81-26ba680fc4ff tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.440959 Apr 12 02:49:15 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 373/1105] 10.0.0.210 () {64 vars in 1232 bytes} [Wed Apr 12 02:49:15 2023] POST /compute/v2.1/os-volumes => generated 345 bytes in 443 msecs (HTTP/1.1 200) 10 headers in 447 bytes (1 switches on core 0) Apr 12 02:49:15 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-5e2969c6-15d1-46b0-b7c6-957d20ccc0f1 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb203093e20>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:15 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-5e2969c6-15d1-46b0-b7c6-957d20ccc0f1 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 200 len: 345 microversion: 2.1 time: 0.324510 Apr 12 02:49:15 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 363/1106] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:15 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 345 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:16 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-187736b2-317c-4413-b395-a71910fb171b tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb2030b6440>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-187736b2-317c-4413-b395-a71910fb171b tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 200 len: 346 microversion: 2.1 time: 0.171173 Apr 12 02:49:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 371/1107] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:16 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 346 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-e10f699a-49e5-4b8c-812a-8b2708b61954 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Action: 'create', calling method: .version_select at 0x7fb20308a440>, body: {"snapshot": {"volume_id": "182a8b7e-a92d-49d1-9776-bc679311157b", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1547106329"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-e10f699a-49e5-4b8c-812a-8b2708b61954 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.153528 Apr 12 02:49:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 374/1108] 10.0.0.210 () {64 vars in 1239 bytes} [Wed Apr 12 02:49:17 2023] POST /compute/v2.1/os-snapshots => generated 287 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:17 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-7c330c18-1758-4598-a47f-17173b830b40 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb2032cad40>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:17 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-7c330c18-1758-4598-a47f-17173b830b40 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 200 len: 287 microversion: 2.1 time: 0.037484 Apr 12 02:49:17 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 364/1109] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:49:17 2023] GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:17 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f0a2e9b4-8cf6-4a73-b34c-b0da0a0f354f tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb20349d7e0>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:17 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f0a2e9b4-8cf6-4a73-b34c-b0da0a0f354f tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.037462 Apr 12 02:49:17 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 372/1110] 10.0.0.210 () {62 vars in 1217 bytes} [Wed Apr 12 02:49:17 2023] GET /compute/v2.1/os-snapshots => generated 290 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:17 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-af7d5efa-dbb1-4fdf-9846-216a3dd02336 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb203ed0b80>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:17 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-af7d5efa-dbb1-4fdf-9846-216a3dd02336 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 200 len: 287 microversion: 2.1 time: 0.033599 Apr 12 02:49:17 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 375/1111] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:49:17 2023] GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 287 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:18 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-24fae726-9de5-4ee5-85dd-4222a07e0611 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb203091900>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:18 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-24fae726-9de5-4ee5-85dd-4222a07e0611 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 200 len: 288 microversion: 2.1 time: 0.036989 Apr 12 02:49:18 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 365/1112] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:49:18 2023] GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 288 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:18 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-10f63bbd-d486-4f6d-8e91-281e6b52088c tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb20349c550>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:18 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-10f63bbd-d486-4f6d-8e91-281e6b52088c tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 202 len: 0 microversion: 2.1 time: 0.065291 Apr 12 02:49:18 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 373/1113] 10.0.0.210 () {62 vars in 1331 bytes} [Wed Apr 12 02:49:18 2023] DELETE /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 0 bytes in 67 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:49:18 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0fc1c7e9-2961-4717-9809-79b7f305a88f tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb20343b910>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:18 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0fc1c7e9-2961-4717-9809-79b7f305a88f tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 200 len: 287 microversion: 2.1 time: 0.037613 Apr 12 02:49:18 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 376/1114] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:49:18 2023] GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99899cfe-63be-462c-972b-b058d0cb96fe tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb2032cb910>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-99899cfe-63be-462c-972b-b058d0cb96fe tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] HTTP exception thrown: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. Apr 12 02:49:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99899cfe-63be-462c-972b-b058d0cb96fe tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Request method failure captured: Apr 12 02:49:19 user devstack@n-api.service[72505]: request: GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 HTTP/1.1 Apr 12 02:49:19 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:49:19 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:49:19 user devstack@n-api.service[72505]: Connection: close Apr 12 02:49:19 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:49:19 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:49:19 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:49:19 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:49:19 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhwqUeWSi1Qbfag1zIkekkENW2mRHQRwUSpbb78rn7RwjaNE3Vbo0XZKJ3lrVhEQfRTXqIkYm6aCajRpdsF3P1jsckVJLwwrHeG3emVfApX7vUfoslgUdvTfD9uxDM74q4qkqdbP0JtzgBgjPIp9q8N58YgEpFwJOTKpkmfDDAadiVA Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Project-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Tenant-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:19 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:19 user devstack@n-api.service[72505]: X-User: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:19 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:49:19 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:49:19 user devstack@n-api.service[72505]: X-User-Id: 4181aa81078049618e140b31da91f4df Apr 12 02:49:19 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:19 user devstack@n-api.service[72505]: method: Apr 12 02:49:19 user devstack@n-api.service[72505]: exception: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. Apr 12 02:49:19 user devstack@n-api.service[72505]: action_args: {'id': '5a269c55-3a61-436e-999e-fa6c44399de4'} Apr 12 02:49:19 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 174, in get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. (HTTP 404) (Request-ID: req-b0237bd9-6089-43c2-bc94-15280abc3f61) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volume_snapshots.py", line 174, in get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. Apr 12 02:49:19 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:19 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-99899cfe-63be-462c-972b-b058d0cb96fe tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Returning 404 to user: Snapshot 5a269c55-3a61-436e-999e-fa6c44399de4 could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:19 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-99899cfe-63be-462c-972b-b058d0cb96fe tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4" status: 404 len: 111 microversion: 2.1 time: 0.053952 Apr 12 02:49:19 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 366/1115] 10.0.0.210 () {62 vars in 1328 bytes} [Wed Apr 12 02:49:19 2023] GET /compute/v2.1/os-snapshots/5a269c55-3a61-436e-999e-fa6c44399de4 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:49:19 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-7422cf99-889a-4a54-952f-14c030938aea tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb2030b5a20>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:19 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-7422cf99-889a-4a54-952f-14c030938aea tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 202 len: 0 microversion: 2.1 time: 0.066139 Apr 12 02:49:19 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 374/1116] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:49:19 2023] DELETE /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:49:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:19 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:19 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-27af16e1-d82b-4619-92d8-cdc69ed15b18 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb2047f2b00>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:19 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-27af16e1-d82b-4619-92d8-cdc69ed15b18 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 200 len: 345 microversion: 2.1 time: 0.051404 Apr 12 02:49:19 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 377/1117] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:19 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 345 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:20 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-aee165e2-e8d1-400c-8228-509664668a22 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb203090ca0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:20 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-aee165e2-e8d1-400c-8228-509664668a22 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 200 len: 345 microversion: 2.1 time: 0.049004 Apr 12 02:49:20 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 367/1118] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:20 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 345 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-dbb28c41-56e1-4b7d-9318-6def28d24e70 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb202fb8280>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:21 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-dbb28c41-56e1-4b7d-9318-6def28d24e70 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] HTTP exception thrown: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-dbb28c41-56e1-4b7d-9318-6def28d24e70 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Request method failure captured: Apr 12 02:49:21 user devstack@n-api.service[72507]: request: GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b HTTP/1.1 Apr 12 02:49:21 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:49:21 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:49:21 user devstack@n-api.service[72507]: Connection: close Apr 12 02:49:21 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:49:21 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:49:21 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:49:21 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:49:21 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhwqUeWSi1Qbfag1zIkekkENW2mRHQRwUSpbb78rn7RwjaNE3Vbo0XZKJ3lrVhEQfRTXqIkYm6aCajRpdsF3P1jsckVJLwwrHeG3emVfApX7vUfoslgUdvTfD9uxDM74q4qkqdbP0JtzgBgjPIp9q8N58YgEpFwJOTKpkmfDDAadiVA Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Project-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Tenant-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72507]: X-User: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72507]: X-User-Id: 4181aa81078049618e140b31da91f4df Apr 12 02:49:21 user devstack@n-api.service[72507]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72507]: method: Apr 12 02:49:21 user devstack@n-api.service[72507]: exception: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72507]: action_args: {'id': '182a8b7e-a92d-49d1-9776-bc679311157b'} Apr 12 02:49:21 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. (HTTP 404) (Request-ID: req-61f18e02-8951-4fb4-bf7e-7a24ec01aaac) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-dbb28c41-56e1-4b7d-9318-6def28d24e70 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Returning 404 to user: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:21 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-dbb28c41-56e1-4b7d-9318-6def28d24e70 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 404 len: 109 microversion: 2.1 time: 0.039648 Apr 12 02:49:21 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 375/1119] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:21 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:49:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-964cf8d8-3b91-432a-b8fc-7eda37eca92d tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb20343a4d0>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.wsgi [None req-964cf8d8-3b91-432a-b8fc-7eda37eca92d tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] HTTP exception thrown: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-964cf8d8-3b91-432a-b8fc-7eda37eca92d tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Request method failure captured: Apr 12 02:49:21 user devstack@n-api.service[72506]: request: DELETE /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b HTTP/1.1 Apr 12 02:49:21 user devstack@n-api.service[72506]: Accept: application/json Apr 12 02:49:21 user devstack@n-api.service[72506]: Accept-Encoding: identity Apr 12 02:49:21 user devstack@n-api.service[72506]: Connection: close Apr 12 02:49:21 user devstack@n-api.service[72506]: Content-Length: 0 Apr 12 02:49:21 user devstack@n-api.service[72506]: Content-Type: application/json Apr 12 02:49:21 user devstack@n-api.service[72506]: Host: 10.0.0.210 Apr 12 02:49:21 user devstack@n-api.service[72506]: Openstack-System-Scope: None Apr 12 02:49:21 user devstack@n-api.service[72506]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Auth-Token: gAAAAABkNhwqUeWSi1Qbfag1zIkekkENW2mRHQRwUSpbb78rn7RwjaNE3Vbo0XZKJ3lrVhEQfRTXqIkYm6aCajRpdsF3P1jsckVJLwwrHeG3emVfApX7vUfoslgUdvTfD9uxDM74q4qkqdbP0JtzgBgjPIp9q8N58YgEpFwJOTKpkmfDDAadiVA Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Domain-Id: None Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Domain-Name: None Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Identity-Status: Confirmed Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Is-Admin-Project: True Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Project-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Project-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Project-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Role: member,reader Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Roles: member,reader Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Tenant-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72506]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72506]: X-User: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72506]: X-User-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72506]: X-User-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72506]: X-User-Id: 4181aa81078049618e140b31da91f4df Apr 12 02:49:21 user devstack@n-api.service[72506]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72506]: method: Apr 12 02:49:21 user devstack@n-api.service[72506]: exception: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72506]: action_args: {'id': '182a8b7e-a92d-49d1-9776-bc679311157b'} Apr 12 02:49:21 user devstack@n-api.service[72506]: {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. (HTTP 404) (Request-ID: req-7ce31b2b-c171-4a46-a7a2-a964ad402b2e) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self._delete(loc) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 313, in _delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 229, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72506]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-964cf8d8-3b91-432a-b8fc-7eda37eca92d tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Returning 404 to user: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. {{(pid=72506) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:21 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-964cf8d8-3b91-432a-b8fc-7eda37eca92d tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 404 len: 109 microversion: 2.1 time: 0.041422 Apr 12 02:49:21 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 378/1120] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:49:21 2023] DELETE /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:49:21 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=5,nova_api:DELETE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:21 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:49:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0a9622b1-0e3f-4e98-a40d-7abcce5dee39 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Calling method '.version_select at 0x7fb203091ea0>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-0a9622b1-0e3f-4e98-a40d-7abcce5dee39 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] HTTP exception thrown: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0a9622b1-0e3f-4e98-a40d-7abcce5dee39 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Request method failure captured: Apr 12 02:49:21 user devstack@n-api.service[72505]: request: GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b HTTP/1.1 Apr 12 02:49:21 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:49:21 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:49:21 user devstack@n-api.service[72505]: Connection: close Apr 12 02:49:21 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:49:21 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:49:21 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:49:21 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:49:21 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhwqUeWSi1Qbfag1zIkekkENW2mRHQRwUSpbb78rn7RwjaNE3Vbo0XZKJ3lrVhEQfRTXqIkYm6aCajRpdsF3P1jsckVJLwwrHeG3emVfApX7vUfoslgUdvTfD9uxDM74q4qkqdbP0JtzgBgjPIp9q8N58YgEpFwJOTKpkmfDDAadiVA Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Project-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/a37fe5d87fa04414bb98bb4da67aba3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Tenant-Id: a37fe5d87fa04414bb98bb4da67aba3f Apr 12 02:49:21 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1278392788 Apr 12 02:49:21 user devstack@n-api.service[72505]: X-User: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:49:21 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:49:21 user devstack@n-api.service[72505]: X-User-Id: 4181aa81078049618e140b31da91f4df Apr 12 02:49:21 user devstack@n-api.service[72505]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1278392788-project-member Apr 12 02:49:21 user devstack@n-api.service[72505]: method: Apr 12 02:49:21 user devstack@n-api.service[72505]: exception: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72505]: action_args: {'id': '182a8b7e-a92d-49d1-9776-bc679311157b'} Apr 12 02:49:21 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. (HTTP 404) (Request-ID: req-7f3c3e2e-2678-4edc-9d82-7a11a70e43f8) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 503, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi context, microversion=microversion).volumes.get(volume_id) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/base.py", line 297, in _get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 220, in get Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 211, in _cs_request Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/cinderclient/client.py", line 197, in request Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. Apr 12 02:49:21 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:49:21 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0a9622b1-0e3f-4e98-a40d-7abcce5dee39 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] Returning 404 to user: Volume 182a8b7e-a92d-49d1-9776-bc679311157b could not be found. {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:49:21 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0a9622b1-0e3f-4e98-a40d-7abcce5dee39 tempest-VolumesSnapshotsTestJSON-1278392788 tempest-VolumesSnapshotsTestJSON-1278392788-project-member] 10.0.0.210 "GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b" status: 404 len: 109 microversion: 2.1 time: 0.038240 Apr 12 02:49:21 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 368/1121] 10.0.0.210 () {62 vars in 1322 bytes} [Wed Apr 12 02:49:21 2023] GET /compute/v2.1/os-volumes/182a8b7e-a92d-49d1-9776-bc679311157b => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:49:42 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:49:42 user devstack@n-api.service[72507]: Wed Apr 12 02:49:42 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs (ip 10.0.0.210) !!! Apr 12 02:49:42 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:49:42 user devstack@n-api.service[72507]: Wed Apr 12 02:49:42 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs (ip 10.0.0.210) !!! Apr 12 02:49:42 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a9a11824-81a8-43fc-b03f-585ef80b435e tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'create', calling method: .version_select at 0x7fb202fb9090>, body: {"keypair": {"name": "tempest-TestStampPattern-1893093837", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHpyzmcRX9rlNwf8L3npzKG+KhZsGBYg2FMRQESss2fLWTR7mB/j7XujY+I5ifJXOZVNKSQj7zwTGJjBzU1DUaSm2KRtc1OjWfWfBP++qIhir5IuETJWfPk9BYw0ygsoOw=="}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:42 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-a9a11824-81a8-43fc-b03f-585ef80b435e tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for project 6594c28fbe004babb03de341f082b858. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:49:42 user devstack@n-api.service[72507]: DEBUG nova.quota [None req-a9a11824-81a8-43fc-b03f-585ef80b435e tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for user e5ed29e7a68045d1bf0b98f416adcb2f and project 6594c28fbe004babb03de341f082b858. Resources: {'key_pairs'} {{(pid=72507) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:49:42 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a9a11824-81a8-43fc-b03f-585ef80b435e tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/os-keypairs" status: 200 len: 395 microversion: 2.1 time: 0.265121 Apr 12 02:49:42 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 376/1122] 10.0.0.210 () {64 vars in 1236 bytes} [Wed Apr 12 02:49:42 2023] POST /compute/v2.1/os-keypairs => generated 395 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:45 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestStampPattern-server-375589421", "imageRef": "16f93c82-82f4-4843-b632-b6b6fe0b6c16", "flavorRef": "42", "key_name": "tempest-TestStampPattern-1893093837", "security_groups": [{"name": "tempest-secgroup-smoke-1072920247"}], "networks": [{"uuid": "ee480d6c-5126-42d8-ac77-8c6afc6a1248"}]}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:45 user devstack@n-api.service[72506]: DEBUG nova.network.neutron [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] validate_networks() for [('ee480d6c-5126-42d8-ac77-8c6afc6a1248', None, None, None, None, None)] {{(pid=72506) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Flavor limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Image limits 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Flavor pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Image pref 0:0:0 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.virt.hardware [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72506) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for project 6594c28fbe004babb03de341f082b858. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.quota [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for user e5ed29e7a68045d1bf0b98f416adcb2f and project 6594c28fbe004babb03de341f082b858. Resources: {'cores', 'instances', 'ram'} {{(pid=72506) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Going to run 1 instances... {{(pid=72506) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:49:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='16f93c82-82f4-4843-b632-b6b6fe0b6c16',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72506) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:49:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-b9a7fd6a-32be-4292-a956-1899a20f70b2 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.723902 Apr 12 02:49:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 379/1123] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:49:45 2023] POST /compute/v2.1/servers => generated 404 bytes in 725 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:49:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-b698425d-40e4-43af-8e04-736cbf6bb26f tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:49:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-b698425d-40e4-43af-8e04-736cbf6bb26f tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:49:46 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-b698425d-40e4-43af-8e04-736cbf6bb26f tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:49:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-b698425d-40e4-43af-8e04-736cbf6bb26f tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 200 len: 1236 microversion: 2.1 time: 0.089647 Apr 12 02:49:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 369/1124] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:49:46 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 1236 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:49:48 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:48 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] Creating event network-changed:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:49:48 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-97edf491-15c8-4626-afaa-8114bb406bdb req-f39bf3de-f288-48bb-9c7a-a62002d24e00 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032472 Apr 12 02:49:48 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 377/1125] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:49:48 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:50 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "name": "network-vif-plugged", "status": "completed", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:50 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] Creating event network-vif-plugged:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:49:50 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-c6288d17-1d2e-4e18-b61e-1c05fed9713a req-bd8d0588-1ef7-49c2-9606-3fd9a29c0dcb service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087574 Apr 12 02:49:50 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 380/1126] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:49:50 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:49:52 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "name": "network-vif-plugged", "status": "completed", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:49:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:49:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:49:52 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:49:53 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] Creating event network-vif-plugged:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:49:53 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-8e53744e-82eb-47e9-b90b-1c17398c32e3 req-9cba109f-9fc5-4ce9-8f61-f7a6cc05117e service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030191 Apr 12 02:49:53 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 370/1127] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:49:52 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=13,nova_api:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:50:36 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:50:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-fd0d9e2a-f182-4803-8b6b-87c41bf1f7a9 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 200 len: 1539 microversion: 2.1 time: 0.189776 Apr 12 02:50:36 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 378/1128] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:50:36 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 1539 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell0:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=3 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:37 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:51:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-f0c416a7-06eb-4cfe-aa1b-fd24cff2db5b tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 200 len: 1539 microversion: 2.1 time: 0.189180 Apr 12 02:51:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 381/1129] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:51:36 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 1539 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:38 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:51:38 user devstack@n-api.service[72505]: Wed Apr 12 02:51:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] Creating event network-changed:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:51:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-7bd99579-3b2b-4f04-8af5-e5422899073f req-719cde01-76bc-4081-a4e1-bf611be8d50a service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048740 Apr 12 02:51:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 371/1130] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:51:38 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "029c2071-8738-4e0c-84be-af996dcbe9f4"}} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-dd3cd18e-80c6-44e8-bb73-f3fb1c9002b8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.326605 Apr 12 02:51:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 379/1131] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:51:38 2023] POST /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments => generated 194 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:46 user devstack@n-api.service[72506]: DEBUG nova.objects.instance [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lazy-loading 'services' on Instance uuid 91ef1049-cfc1-4ff7-bc79-f17897c49de1 {{(pid=72506) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:51:46 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-0dc2728d-82d1-44d8-ac83-fa70f7ce3ea7 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments/029c2071-8738-4e0c-84be-af996dcbe9f4" status: 202 len: 0 microversion: 2.1 time: 0.181076 Apr 12 02:51:46 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 382/1132] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:51:46 2023] DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments/029c2071-8738-4e0c-84be-af996dcbe9f4 => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:51:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:51:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-e733eaae-a267-41b6-bd17-058ec0deea6d tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053025 Apr 12 02:51:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 372/1133] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:51:46 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:36 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:36 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-f072f871-7f93-4783-93a7-dccc040e173a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.061405 Apr 12 02:52:36 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 380/1134] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:52:36 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments => generated 25 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestStampPatternsnapshot-752374696"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:52:38 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:52:38 user devstack@n-api.service[72506]: Wed Apr 12 02:52:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/action (ip 10.0.0.210) !!! Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=4 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=15,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-84c8536e-31aa-40f6-a900-bf029f372d63 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/action" status: 202 len: 0 microversion: 2.1 time: 0.990280 Apr 12 02:52:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 383/1135] 10.0.0.210 () {64 vars in 1355 bytes} [Wed Apr 12 02:52:38 2023] POST /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/action => generated 0 bytes in 992 msecs (HTTP/1.1 202) 10 headers in 449 bytes (1 switches on core 0) Apr 12 02:52:45 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestStampPattern-server-56434730", "imageRef": "b0a853d4-2140-47ec-8d81-5ef1133ca1cb", "flavorRef": "42", "key_name": "tempest-TestStampPattern-1893093837", "security_groups": [{"name": "tempest-secgroup-smoke-1072920247"}], "networks": [{"uuid": "ee480d6c-5126-42d8-ac77-8c6afc6a1248"}]}} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:52:45 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:45 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=8 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:45 user devstack@n-api.service[72505]: DEBUG nova.network.neutron [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] validate_networks() for [('ee480d6c-5126-42d8-ac77-8c6afc6a1248', None, None, None, None, None)] {{(pid=72505) validate_networks /opt/stack/nova/nova/network/neutron.py:2636}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Flavor limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Image limits 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Flavor pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Image pref 0:0:0 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.virt.hardware [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=72505) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for project 6594c28fbe004babb03de341f082b858. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:393}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.quota [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Getting quotas for user e5ed29e7a68045d1bf0b98f416adcb2f and project 6594c28fbe004babb03de341f082b858. Resources: {'cores', 'instances', 'ram'} {{(pid=72505) _get_quotas /opt/stack/nova/nova/quota.py:383}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Going to run 1 instances... {{(pid=72505) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Apr 12 02:52:46 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b0a853d4-2140-47ec-8d81-5ef1133ca1cb',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=72505) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Apr 12 02:52:46 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3a602caf-ffd0-4e11-bc6f-df16426bd475 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.709307 Apr 12 02:52:46 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 373/1136] 10.0.0.210 () {64 vars in 1224 bytes} [Wed Apr 12 02:52:45 2023] POST /compute/v2.1/servers => generated 404 bytes in 711 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Apr 12 02:52:46 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-e3bf3dbb-419f-4d7f-87cc-21bb9d8949a6 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:52:46 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-e3bf3dbb-419f-4d7f-87cc-21bb9d8949a6 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:52:46 user devstack@n-api.service[72507]: DEBUG nova.policy [None req-e3bf3dbb-419f-4d7f-87cc-21bb9d8949a6 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72507) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:52:46 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-e3bf3dbb-419f-4d7f-87cc-21bb9d8949a6 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 200 len: 1235 microversion: 2.1 time: 0.089133 Apr 12 02:52:46 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 381/1137] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:52:46 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 1235 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=13,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:49 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] Creating event network-changed:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:52:49 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-5fc69c8c-51e3-47f1-a385-f63c4805edae req-97e7c4e8-50be-48eb-81b6-09af68d8457b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038037 Apr 12 02:52:49 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 384/1138] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:52:49 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:52:51 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "name": "network-vif-plugged", "status": "completed", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:52:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:51 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] Creating event network-vif-plugged:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:52:51 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-d92ac27d-c98e-4c5b-8d87-b2bd7db7812b req-3ca97e15-fb69-416c-a930-936b37cf5f69 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033169 Apr 12 02:52:51 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 374/1139] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:52:51 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:52:53 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "name": "network-vif-plugged", "status": "completed", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:52:53 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:52:53 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:52:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] Creating event network-vif-plugged:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:52:53 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-7be9eefc-5720-4a39-958e-d40c8f936cda req-92abd0a0-0a6c-4ad3-9a15-b836310b609c service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039773 Apr 12 02:52:53 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 382/1140] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:52:53 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:53:36 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:53:36 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-39ac0da9-f7ed-463b-9dbb-fa3fe19c7bec tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 200 len: 1537 microversion: 2.1 time: 0.237237 Apr 12 02:53:36 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 385/1141] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:53:36 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 1537 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell0:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=17,nova_api:INSERT=3 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=4 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:36 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:37 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:54:37 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-3dc9eca6-8131-4c2d-81f9-2355b56b6333 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 200 len: 1537 microversion: 2.1 time: 0.252917 Apr 12 02:54:37 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 375/1142] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:54:36 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 1537 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell0:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=2 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:38 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:54:38 user devstack@n-api.service[72507]: Wed Apr 12 02:54:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:54:38 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:54:38 user devstack@n-api.service[72507]: Wed Apr 12 02:54:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.0.0.210) !!! Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:38 user devstack@n-api.service[72506]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] Creating event network-changed:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:54:38 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:54:38 user devstack@n-api.service[72506]: Wed Apr 12 02:54:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments (ip 10.0.0.210) !!! Apr 12 02:54:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-c23d1c64-510d-4f97-abca-449d23fa1e8f req-0f788a0f-c7a2-4cc5-9b2c-7777b79d6a37 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100536 Apr 12 02:54:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 383/1143] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:54:38 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7459b83b-dbd5-45c0-8dcc-0696b2768784"}} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:39 user devstack@n-api.service[72506]: INFO oslo.messaging._drivers.impl_rabbit [-] [c600c65c-e58a-47df-9fb4-3ffffd6f5667] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 40296. Apr 12 02:54:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-9cdb56a5-7632-484e-bf38-fb6812186166 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "POST /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.129055 Apr 12 02:54:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 386/1144] 10.0.0.210 () {64 vars in 1400 bytes} [Wed Apr 12 02:54:38 2023] POST /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments => generated 194 bytes in 1131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=6 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:47 user devstack@n-api.service[72505]: DEBUG nova.objects.instance [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lazy-loading 'services' on Instance uuid c19065ba-2de8-4aa3-ba51-458dc5dd18b0 {{(pid=72505) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Apr 12 02:54:47 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-0994dcc8-2c3c-48bd-a455-66965db6fa42 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments/7459b83b-dbd5-45c0-8dcc-0696b2768784" status: 202 len: 0 microversion: 2.1 time: 0.189319 Apr 12 02:54:47 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 376/1145] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:54:47 2023] DELETE /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments/7459b83b-dbd5-45c0-8dcc-0696b2768784 => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:54:47 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:54:47 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-a1b52c65-925e-41f3-81b2-972699d6dfa4 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044740 Apr 12 02:54:47 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 384/1146] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:54:47 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=2 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:37 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:37 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-afc99549-5d82-4a2b-b305-c91276f66611 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057914 Apr 12 02:55:37 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 387/1147] 10.0.0.210 () {62 vars in 1379 bytes} [Wed Apr 12 02:55:37 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Apr 12 02:55:38 user devstack@n-api.service[72505]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] AMQP server on 10.0.0.210:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Apr 12 02:55:38 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Apr 12 02:55:38 user devstack@n-api.service[72505]: Wed Apr 12 02:55:38 2023 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 (ip 10.0.0.210) !!! Apr 12 02:55:38 user devstack@n-api.service[72507]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] AMQP server on 10.0.0.210:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=2 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=8 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:38 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] Creating event network-changed:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:55:38 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-9097e2b9-cb5c-4830-8cc5-49389578f1c2 req-75d7dc17-6be9-44b9-920c-5ad348b8d656 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040679 Apr 12 02:55:38 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 385/1148] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:55:38 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:55:38 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Going to try to terminate instance {{(pid=72505) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:55:38 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-959c1209-091f-45ad-ae1d-81bd0d5aad21 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 204 len: 0 microversion: 2.1 time: 0.202686 Apr 12 02:55:38 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 377/1149] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:55:38 2023] DELETE /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:38 user devstack@n-api.service[72506]: DEBUG nova.policy [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72506) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:55:39 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-5e8485e8-a2db-454b-9d2c-2dbf1a3e3c01 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 200 len: 1661 microversion: 2.1 time: 0.284234 Apr 12 02:55:39 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 388/1150] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:55:38 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 1661 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:55:39 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "name": "network-vif-unplugged", "status": "completed", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:55:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:39 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] Creating event network-vif-unplugged:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:55:39 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-a3c67a42-6e7a-443b-834e-c3a1aec73c82 req-706d3523-8215-4b04-bffa-cf5c18e5cd18 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.080929 Apr 12 02:55:39 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 386/1151] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:55:39 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:55:40 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "name": "network-vif-deleted", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:40 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] Creating event network-vif-deleted:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:55:40 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-f3ff44a1-81e3-406d-a13e-f0aabe19e51c req-66e82d3f-45f5-46dd-9663-5930f67651b5 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042742 Apr 12 02:55:40 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 378/1152] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:55:40 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:55:41 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c19065ba-2de8-4aa3-ba51-458dc5dd18b0", "name": "network-vif-plugged", "status": "completed", "tag": "94b1f9ed-97f2-47b6-a547-86c7bda32221"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:55:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:55:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:55:41 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:55:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] Creating event network-vif-plugged:94b1f9ed-97f2-47b6-a547-86c7bda32221 for instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 on user Apr 12 02:55:41 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-696eed37-4608-446d-8bc9-f9b6d6d5716c req-ee67d5c8-03e7-4a6a-916f-7718b2e1dc63 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041533 Apr 12 02:55:41 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 389/1153] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:55:41 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=4 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: c19065ba-2de8-4aa3-ba51-458dc5dd18b0] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:29 user devstack@n-api.service[72507]: INFO oslo.messaging._drivers.impl_rabbit [-] [a2b755db-5213-425e-abce-72adb06a9560] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 47186. Apr 12 02:56:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] HTTP exception thrown: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Request method failure captured: Apr 12 02:56:29 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 HTTP/1.1 Apr 12 02:56:29 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:56:29 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:56:29 user devstack@n-api.service[72507]: Connection: close Apr 12 02:56:29 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:56:29 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:56:29 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:56:29 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:56:29 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhxAXDCDjOryWz6o3b-VlyZqCVyI9v2kARn1NHNbaTSouGV-xKLYfjmDqQzxfLz_8GRWghKGRD9Vu3V2oGRNYKiowSthqiIFpHpmWtcngj7fj3FoN6BsOs0gifimksN0d8Dg28EMgYe1cpOp9gzWnHrnMvFqqM-UYcXSx_hNJgPi8P4 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Project-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestStampPattern-1035046562 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6594c28fbe004babb03de341f082b858"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Tenant: tempest-TestStampPattern-1035046562 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Tenant-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestStampPattern-1035046562 Apr 12 02:56:29 user devstack@n-api.service[72507]: X-User: tempest-TestStampPattern-1035046562-project-member Apr 12 02:56:29 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:56:29 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:56:29 user devstack@n-api.service[72507]: X-User-Id: e5ed29e7a68045d1bf0b98f416adcb2f Apr 12 02:56:29 user devstack@n-api.service[72507]: X-User-Name: tempest-TestStampPattern-1035046562-project-member Apr 12 02:56:29 user devstack@n-api.service[72507]: method: > Apr 12 02:56:29 user devstack@n-api.service[72507]: exception: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. Apr 12 02:56:29 user devstack@n-api.service[72507]: action_args: {'id': 'c19065ba-2de8-4aa3-ba51-458dc5dd18b0'} Apr 12 02:56:29 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. Apr 12 02:56:29 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:56:29 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Returning 404 to user: Instance c19065ba-2de8-4aa3-ba51-458dc5dd18b0 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:56:29 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-3a620122-aa35-4456-b858-e9a003051622 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0" status: 404 len: 111 microversion: 2.1 time: 0.045535 Apr 12 02:56:29 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 387/1154] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:56:29 2023] GET /compute/v2.1/servers/c19065ba-2de8-4aa3-ba51-458dc5dd18b0 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG dbcounter [-] [72505] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72505) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:56:31 user devstack@n-api.service[72505]: INFO oslo.messaging._drivers.impl_rabbit [-] [4aa86c71-2b1f-45b6-8d2a-625b297c6ba1] Reconnected to AMQP server on 10.0.0.210:5672 via [amqp] client with port 45754. Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.wsgi [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] HTTP exception thrown: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 is not attached to volume 029c2071-8738-4e0c-84be-af996dcbe9f4 Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Request method failure captured: Apr 12 02:56:31 user devstack@n-api.service[72505]: request: DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments/029c2071-8738-4e0c-84be-af996dcbe9f4 HTTP/1.1 Apr 12 02:56:31 user devstack@n-api.service[72505]: Accept: application/json Apr 12 02:56:31 user devstack@n-api.service[72505]: Accept-Encoding: identity Apr 12 02:56:31 user devstack@n-api.service[72505]: Connection: close Apr 12 02:56:31 user devstack@n-api.service[72505]: Content-Length: 0 Apr 12 02:56:31 user devstack@n-api.service[72505]: Content-Type: application/json Apr 12 02:56:31 user devstack@n-api.service[72505]: Host: 10.0.0.210 Apr 12 02:56:31 user devstack@n-api.service[72505]: Openstack-System-Scope: None Apr 12 02:56:31 user devstack@n-api.service[72505]: User-Agent: python-urllib3/1.26.15 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Auth-Token: gAAAAABkNhxAXDCDjOryWz6o3b-VlyZqCVyI9v2kARn1NHNbaTSouGV-xKLYfjmDqQzxfLz_8GRWghKGRD9Vu3V2oGRNYKiowSthqiIFpHpmWtcngj7fj3FoN6BsOs0gifimksN0d8Dg28EMgYe1cpOp9gzWnHrnMvFqqM-UYcXSx_hNJgPi8P4 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Domain-Id: None Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Domain-Name: None Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Identity-Status: Confirmed Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Is-Admin-Project: True Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Project-Domain-Id: default Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Project-Domain-Name: Default Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Project-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Project-Name: tempest-TestStampPattern-1035046562 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Role: member,reader Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Roles: member,reader Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6594c28fbe004babb03de341f082b858"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Tenant: tempest-TestStampPattern-1035046562 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Tenant-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-Tenant-Name: tempest-TestStampPattern-1035046562 Apr 12 02:56:31 user devstack@n-api.service[72505]: X-User: tempest-TestStampPattern-1035046562-project-member Apr 12 02:56:31 user devstack@n-api.service[72505]: X-User-Domain-Id: default Apr 12 02:56:31 user devstack@n-api.service[72505]: X-User-Domain-Name: Default Apr 12 02:56:31 user devstack@n-api.service[72505]: X-User-Id: e5ed29e7a68045d1bf0b98f416adcb2f Apr 12 02:56:31 user devstack@n-api.service[72505]: X-User-Name: tempest-TestStampPattern-1035046562-project-member Apr 12 02:56:31 user devstack@n-api.service[72505]: method: > Apr 12 02:56:31 user devstack@n-api.service[72505]: exception: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 is not attached to volume 029c2071-8738-4e0c-84be-af996dcbe9f4 Apr 12 02:56:31 user devstack@n-api.service[72505]: action_args: {'server_id': '91ef1049-cfc1-4ff7-bc79-f17897c49de1', 'id': '029c2071-8738-4e0c-84be-af996dcbe9f4'} Apr 12 02:56:31 user devstack@n-api.service[72505]: {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.10/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 029c2071-8738-4e0c-84be-af996dcbe9f4. Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 is not attached to volume 029c2071-8738-4e0c-84be-af996dcbe9f4 Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 is not attached to volume 029c2071-8738-4e0c-84be-af996dcbe9f4 Apr 12 02:56:31 user devstack@n-api.service[72505]: ERROR nova.api.openstack.wsgi Apr 12 02:56:31 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Returning 404 to user: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 is not attached to volume 029c2071-8738-4e0c-84be-af996dcbe9f4 {{(pid=72505) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:56:31 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-80c527d3-da49-4556-b723-fe08f11fdd0a tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments/029c2071-8738-4e0c-84be-af996dcbe9f4" status: 404 len: 154 microversion: 2.1 time: 0.128623 Apr 12 02:56:31 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 379/1155] 10.0.0.210 () {62 vars in 1493 bytes} [Wed Apr 12 02:56:31 2023] DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1/os-volume_attachments/029c2071-8738-4e0c-84be-af996dcbe9f4 => generated 154 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=6 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=14 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72506) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] Creating event network-changed:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:56:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-bfe9a358-fe30-4a80-b44b-2f5129646809 req-c8eddef4-9a5d-43d7-8cf4-33e8af02bde8 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032635 Apr 12 02:56:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 388/1156] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:56:32 2023] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:56:32 user devstack@n-api.service[72506]: DEBUG nova.compute.api [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Going to try to terminate instance {{(pid=72506) delete /opt/stack/nova/nova/compute/api.py:2721}} Apr 12 02:56:32 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-6827543e-cd87-403e-ba66-2c8d1bc07db8 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 204 len: 0 microversion: 2.1 time: 0.133429 Apr 12 02:56:32 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 390/1157] 10.0.0.210 () {62 vars in 1316 bytes} [Wed Apr 12 02:56:32 2023] DELETE /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG nova.compute.api [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72505) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72505]: DEBUG nova.policy [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e5ed29e7a68045d1bf0b98f416adcb2f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6594c28fbe004babb03de341f082b858', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=72505) authorize /opt/stack/nova/nova/policy.py:203}} Apr 12 02:56:32 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [None req-1a0c52ba-7861-49c9-b61e-e8e2cc023213 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 200 len: 1664 microversion: 2.1 time: 0.260084 Apr 12 02:56:32 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 380/1158] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:56:32 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 1664 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "name": "network-vif-unplugged", "status": "completed", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:32 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.compute.server_external_events [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] Creating event network-vif-unplugged:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:56:32 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [req-e41673f6-2440-4d19-9969-4922de98ef84 req-69bc9938-edee-4fcc-ae8c-82e6b929e912 service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.029705 Apr 12 02:56:32 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 389/1159] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:56:32 2023] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:56:34 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "name": "network-vif-deleted", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:56:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:34 user devstack@n-api.service[72506]: DEBUG oslo_concurrency.lockutils [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72506) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.compute.server_external_events [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] Creating event network-vif-deleted:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:56:34 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [req-59f2fc07-34f9-462c-a0cf-227db3dd6582 req-d88a3876-8110-4ffb-85f8-aeba5bc6011d service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033237 Apr 12 02:56:34 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 391/1160] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:56:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:56:34 user devstack@n-api.service[72505]: DEBUG nova.api.openstack.wsgi [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "91ef1049-cfc1-4ff7-bc79-f17897c49de1", "name": "network-vif-plugged", "status": "completed", "tag": "55871f74-ac4f-4e6f-86cb-5f593dd8de7d"}]} {{(pid=72505) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Apr 12 02:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:56:34 user devstack@n-api.service[72505]: DEBUG oslo_concurrency.lockutils [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72505) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:56:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.compute.server_external_events [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] Creating event network-vif-plugged:55871f74-ac4f-4e6f-86cb-5f593dd8de7d for instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 on user Apr 12 02:56:34 user devstack@n-api.service[72505]: INFO nova.api.openstack.requestlog [req-a842579d-59c2-42cb-a23b-e08221efda82 req-a48fdc50-9ac0-48d7-bc63-4be030c58a9b service nova] 10.0.0.210 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035588 Apr 12 02:56:34 user devstack@n-api.service[72505]: [pid: 72505|app: 0|req: 381/1161] 10.0.0.210 () {68 vars in 1394 bytes} [Wed Apr 12 02:56:34 2023] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_api:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG dbcounter [-] [72507] Writing DB stats nova_cell1:SELECT=6 {{(pid=72507) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '>' {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG nova.compute.api [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] [instance: 91ef1049-cfc1-4ff7-bc79-f17897c49de1] Fetching instance by UUID {{(pid=72507) get /opt/stack/nova/nova/compute/api.py:2977}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Acquiring lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:404}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:409}} Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG oslo_concurrency.lockutils [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Lock "3f163be6-585e-46c9-b7d4-9054f44c8b1b" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=72507) inner /usr/local/lib/python3.10/dist-packages/oslo_concurrency/lockutils.py:423}} Apr 12 02:57:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.wsgi [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] HTTP exception thrown: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Request method failure captured: Apr 12 02:57:22 user devstack@n-api.service[72507]: request: GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 HTTP/1.1 Apr 12 02:57:22 user devstack@n-api.service[72507]: Accept: application/json Apr 12 02:57:22 user devstack@n-api.service[72507]: Accept-Encoding: identity Apr 12 02:57:22 user devstack@n-api.service[72507]: Connection: close Apr 12 02:57:22 user devstack@n-api.service[72507]: Content-Length: 0 Apr 12 02:57:22 user devstack@n-api.service[72507]: Content-Type: application/json Apr 12 02:57:22 user devstack@n-api.service[72507]: Host: 10.0.0.210 Apr 12 02:57:22 user devstack@n-api.service[72507]: Openstack-System-Scope: None Apr 12 02:57:22 user devstack@n-api.service[72507]: User-Agent: python-urllib3/1.26.15 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Auth-Token: gAAAAABkNhxAXDCDjOryWz6o3b-VlyZqCVyI9v2kARn1NHNbaTSouGV-xKLYfjmDqQzxfLz_8GRWghKGRD9Vu3V2oGRNYKiowSthqiIFpHpmWtcngj7fj3FoN6BsOs0gifimksN0d8Dg28EMgYe1cpOp9gzWnHrnMvFqqM-UYcXSx_hNJgPi8P4 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Domain-Id: None Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Domain-Name: None Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Identity-Status: Confirmed Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Is-Admin-Project: True Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Project-Domain-Id: default Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Project-Domain-Name: Default Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Project-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Project-Name: tempest-TestStampPattern-1035046562 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Role: member,reader Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Roles: member,reader Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Service-Catalog: [{"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2/6594c28fbe004babb03de341f082b858"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/identity"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210/volume/v3/6594c28fbe004babb03de341f082b858"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.0.0.210:9696/networking"}]}] Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Tenant: tempest-TestStampPattern-1035046562 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Tenant-Id: 6594c28fbe004babb03de341f082b858 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-Tenant-Name: tempest-TestStampPattern-1035046562 Apr 12 02:57:22 user devstack@n-api.service[72507]: X-User: tempest-TestStampPattern-1035046562-project-member Apr 12 02:57:22 user devstack@n-api.service[72507]: X-User-Domain-Id: default Apr 12 02:57:22 user devstack@n-api.service[72507]: X-User-Domain-Name: Default Apr 12 02:57:22 user devstack@n-api.service[72507]: X-User-Id: e5ed29e7a68045d1bf0b98f416adcb2f Apr 12 02:57:22 user devstack@n-api.service[72507]: X-User-Name: tempest-TestStampPattern-1035046562-project-member Apr 12 02:57:22 user devstack@n-api.service[72507]: method: > Apr 12 02:57:22 user devstack@n-api.service[72507]: exception: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. Apr 12 02:57:22 user devstack@n-api.service[72507]: action_args: {'id': '91ef1049-cfc1-4ff7-bc79-f17897c49de1'} Apr 12 02:57:22 user devstack@n-api.service[72507]: {{(pid=72507) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2980, in get Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in _get_instance Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2884, in _get_instance_from_cell Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = self._get_server( Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 538, in _process_stack Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi with ResourceExceptionHandler(): Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. Apr 12 02:57:22 user devstack@n-api.service[72507]: ERROR nova.api.openstack.wsgi Apr 12 02:57:22 user devstack@n-api.service[72507]: DEBUG nova.api.openstack.wsgi [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Returning 404 to user: Instance 91ef1049-cfc1-4ff7-bc79-f17897c49de1 could not be found. {{(pid=72507) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Apr 12 02:57:22 user devstack@n-api.service[72507]: INFO nova.api.openstack.requestlog [None req-9735b03b-73f2-41ad-bed1-c9b4f213a369 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1" status: 404 len: 111 microversion: 2.1 time: 0.038810 Apr 12 02:57:22 user devstack@n-api.service[72507]: [pid: 72507|app: 0|req: 390/1162] 10.0.0.210 () {62 vars in 1313 bytes} [Wed Apr 12 02:57:22 2023] GET /compute/v2.1/servers/91ef1049-cfc1-4ff7-bc79-f17897c49de1 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Apr 12 02:57:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:57:25 user devstack@n-api.service[72506]: DEBUG dbcounter [-] [72506] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=72506) stat_writer /usr/local/lib/python3.10/dist-packages/dbcounter.py:117}} Apr 12 02:57:25 user devstack@n-api.service[72506]: DEBUG nova.api.openstack.wsgi [None req-cff34be6-73f3-4a99-87db-bfe64ed77605 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] Calling method '.version_select at 0x7fb20355e440>' {{(pid=72506) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Apr 12 02:57:25 user devstack@n-api.service[72506]: INFO nova.api.openstack.requestlog [None req-cff34be6-73f3-4a99-87db-bfe64ed77605 tempest-TestStampPattern-1035046562 tempest-TestStampPattern-1035046562-project-member] 10.0.0.210 "DELETE /compute/v2.1/os-keypairs/tempest-TestStampPattern-1893093837" status: 202 len: 0 microversion: 2.1 time: 0.050859 Apr 12 02:57:25 user devstack@n-api.service[72506]: [pid: 72506|app: 0|req: 392/1163] 10.0.0.210 () {62 vars in 1325 bytes} [Wed Apr 12 02:57:25 2023] DELETE /compute/v2.1/os-keypairs/tempest-TestStampPattern-1893093837 => generated 0 bytes in 53 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)